Replace the long-lived DOCS_REPO_TOKEN personal access token with
short-lived installation tokens minted per-job from the sencho-token-app
GitHub App (owned by the AnsoCode org, installed on both Sencho and
sencho-docs).
Why this is better than the PAT:
- Scoped per-job to only the repo the job actually touches, not one
token that works everywhere. update-screenshots and release-please
mint against Sencho; sync-docs mints against sencho-docs.
- Scoped per-job to only the permission scopes the job actually needs
(contents:write and sometimes pull-requests:write), not the broader
scope a PAT carries.
- Auto-revoked at job end by the action's post-run step, so a leaked
log line or artifact is useful for at most the remainder of that job.
- Not tied to any individual user account: no more risk of the token
silently dying when the human who minted it leaves or rotates keys.
- Like a PAT but unlike GITHUB_TOKEN, installation tokens DO trigger
downstream workflow runs, so the release-please tag push still
cascades into docker-publish.yml exactly as before.
Jobs migrated:
- release-please.yml: single mint, passed to release-please-action.
- ci.yml update-screenshots: one mint reused by checkout, peter-evans
create-pull-request, and the gh pr merge step.
- ci.yml sync-docs: one mint targeting sencho-docs, fed to the git
clone URL as x-access-token.
DOCS_REPO_TOKEN can be deleted from repo secrets after the first full
release cycle under this migration validates release-please,
update-screenshots, and sync-docs all succeed end-to-end.
actions/create-github-app-token pinned to v3.0.0 at SHA
f8d387b68d61c58ab83c6c016672934102569859, consistent with the
SHA-pinning convention established in the supply-chain hardening PR.
The runtime stage's `apk upgrade --no-cache` layer was being reused from
the buildx gha/registry cache indefinitely, so newly-patched Alpine
packages (e.g. openssl CVE fixes) could sit behind a stale layer until
an unrelated Dockerfile change invalidated it by coincidence. This
surfaced as Trivy hard-failing on CVE-2026-28390 (libcrypto3/libssl3
DoS in CMS) on the github-app-token-migration PR, even though upstream
Alpine had already published 3.5.6-r0 with the fix.
Add an APK_CACHE_BUST build-arg that CI sets to the current UTC date
(YYYY-MM-DD). Each calendar day the arg value changes, which changes
the RUN layer's hash, which forces buildx to re-execute `apk upgrade`
and pick up whatever Alpine has published that day. All three build
invocations (PR docker-validate, release pre-publish scan, release
multi-arch push) pass the same value, so the scan and the published
artifact always share a layer and Trivy never scans stale bits.
Default value `unset` lets local developers `docker build` without
the arg; production CI always supplies the date.
The Playwright browser download (Chrome for Testing + Chrome Headless
Shell + FFmpeg, ~170MB total) is the single largest cost in the
start-app composite action. Observed in a recent E2E run: 4m15s to
download cold vs ~10s on a warm cache. Across weekly cache evictions
and the ~30% of runs that hit a cold cache, that is multiple action
minutes per PR averaged over time, and several minutes of wall-time
regression on any individual cold run.
Split the existing `playwright install --with-deps chromium` step
into two halves:
1. `playwright install-deps chromium` runs unconditionally because
apt packages live under /var and are not preserved by actions/cache.
This is fast (~10s).
2. `playwright install chromium` runs only when the browser cache
under ~/.cache/ms-playwright misses. The cache key is pinned to
the @playwright/test version read from the root package.json, so
a future bump invalidates the cache automatically and the old
browsers are replaced cleanly.
Sharding (the other half of the original optimization plan) was
evaluated and rejected. With ~80s of per-job setup and only ~60s of
test time, 2-way sharding would roughly double action minutes while
saving only ~5s of wall time on warm runs. The math only pencils out
when test time significantly exceeds setup time, which is not the
case for Sencho today. Revisit if the suite grows past ~5 minutes
of pure test time.
The backend CI job already runs `tsc` against backend/src/ and only
marks the job green after build + tests + lint + audit all pass. The
E2E job then checked out the same source and ran the same tsc again
inside the start-app composite.
Upload backend/dist/ as a 1-day retention artifact at the end of the
backend job, download it in the E2E job before invoking the composite,
and gate the composite's "Build backend" step on a new
`skip-backend-build` input. The update-screenshots job (which also
uses the composite but has no upstream backend job) leaves the input
at its default of 'false' and continues to build from source.
Savings are modest (~10-15s per E2E run, on top of what setup-node's
~/.npm cache already buys us) but the real win is the contract:
the dist/ that E2E exercises is now byte-for-byte the same artifact
that the backend job verified, eliminating a whole class of "works
in backend job, breaks in E2E" drift.
Starts the already-loaded localhost/sencho:release-scan image headless
on the runner and polls /api/health for up to 30 seconds. Catches
entrypoint regressions, native module ABI breakage, and first-boot
crashes on the exact artifact that the multi-arch push step below
will republish moments later.
Placed BEFORE the build-push step so a smoke failure does not move
the `latest`, semver, or minor tags on Docker Hub. The image reused
here is the same amd64 artifact Trivy scanned a step earlier, so
there is no extra build or pull on the critical path.
The container is started without --rm so that docker logs in the
trap can surface the stack trace when the container crashes during
boot; the trap force-removes it after capturing logs whether it
exited or is still running.
/api/health is public and returns before any DB, JWT, or Docker
socket work, so no env vars, volume mounts, or bind mounts are
required for the smoke test to be a valid signal that the process
booted and the HTTP server is accepting connections.
Skip the backend, frontend, docker-validate, and e2e jobs on the
chore/refresh-screenshots auto-PR opened by the update-screenshots job.
Screenshot PRs only touch docs/images/ and cannot affect build, lint,
or test surfaces, so running the full PR CI suite on them burns ~6
action minutes per release for zero signal. This mirrors the existing
release-please--branches--main--components--sencho skip.
Also short-circuit sync-docs: when a push to main does not touch any
file under docs/, skip the clone, rsync, and commit steps. rsync
--delete would be a no-op anyway, but spinning up the runner and
cloning sencho-docs still burns ~30s per non-docs merge, which adds
up across multiple merges per day. Requires fetch-depth: 2 so the
detect step can diff HEAD~1 HEAD.
Finally, correct the scan-build comment in docker-publish.yml: the
push-build's amd64 leg reuses layers from the buildkit daemon's
in-memory cache (same job, same daemon), not from the shared registry
buildcache. The cache-from pull is a cold-start fallback for the
first-ever run.
* ci: hard-fail PR and release scans on unacknowledged HIGH/CRITICAL CVEs
Make Trivy a real gate instead of an advisory signal:
- PR CI (docker-validate) no longer uses `continue-on-error: true` on the
Trivy step, so any HIGH/CRITICAL finding not in `.trivyignore` fails the PR.
- The release pipeline (docker-publish.yml) now builds an amd64-only scan
image into the local daemon before the multi-arch push-build, re-runs
Trivy against that exact artifact, and only proceeds to the push if the
scan passes. This closes the gap where a CVE landed between PR merge and
release-time rebuild.
- New `.trivyignore` at repo root is the single source of truth for
acknowledged CVEs across both workflows; it starts empty so the first CI
run surfaces the full list, which we then populate with justifications.
* ci: populate .trivyignore with initial HIGH/CRITICAL acknowledgements
First CI run on the hard-fail Trivy policy surfaced 7 unacknowledged
findings. Each has been reviewed and justified inline:
- 6 CVEs in the statically-linked Go modules inside docker-compose v5.1.1
(github.com/docker/docker, buildkit, otel/sdk x2, grpc). These are
transitively bundled and cannot be bumped without an upstream Compose
rebuild. The grpc CVE is already explicitly acknowledged in the
Dockerfile rationale block at Dockerfile:108-111.
- 1 CVE in picomatch 4.0.3 bundled inside the npm CLI that ships with
node:22-alpine. npm is only invoked at build time against our own
package.json, so the ReDoS vector is not reachable.
Every entry has a revisit trigger (next Compose release or next Alpine
node bump).
Pin the push_to_registry job to a GitHub `production` environment so the
DOCKERHUB_USERNAME and DOCKERHUB_TOKEN credentials can live there
instead of as repo-wide secrets. Any future workflow that tries to push
to Docker Hub without declaring this environment will fail to resolve
the credentials, which is exactly the blast-radius reduction we want.
As a side benefit, adding a required reviewer to the `production`
environment in repo settings now turns every release into a manual
approval gate with zero workflow changes. The secrets were already
moved into the environment ahead of this commit, so the next release
run will pick them up seamlessly.
When an E2E test flakes in CI, the backend and frontend stdout is gone
the moment the step exits. That makes a failed run extremely hard to
triage: you get the Playwright report but nothing from the services
underneath it. Redirect both dev servers to files under the workspace's
ci-logs/ directory and include that directory in the failure-case
artifact upload, so the report and the service logs travel together.
Log files live under $GITHUB_WORKSPACE rather than /tmp because
actions/upload-artifact@v4+ computes a common parent directory across
all path entries. Mixing repo-relative paths with /tmp paths makes '/'
the common root, which v4+ then strips from the artifact, silently
dropping the absolute entries. Keeping everything under the workspace
avoids that footgun.
`stdbuf -oL -eL` forces line-buffered output on both services. Node
stdout is block-buffered when redirected to a regular file, so a crash
or abort can lose the final few hundred lines - exactly the output you
need when debugging a CI failure.
Also remove the duplicated `E2E_USERNAME: admin` / `E2E_PASSWORD:
password123` env blocks from the e2e and update-screenshots jobs.
Those values were already the fallback defaults in e2e/helpers.ts
(`process.env.E2E_USERNAME ?? 'admin'`), so the env blocks were doing
nothing on the default path and created two sources of truth for the
same literals. e2e/helpers.ts remains the canonical source; callers
that want to override the creds still set E2E_USERNAME / E2E_PASSWORD
as job-level env.
Every published Sencho Docker image is now signed with Sigstore cosign
using GitHub Actions keyless OIDC, and ships with an embedded SBOM plus
SLSA provenance attestation attached as OCI referrers on the image.
Users can verify the signature and inspect the attestations themselves
before pulling an image into production.
cosign verify saelix/sencho:<tag> \
--certificate-identity-regexp "https://github.com/AnsoCode/Sencho/.*" \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
The release pipeline now also publishes a new moving minor tag
(saelix/sencho:X.Y, e.g. 0.42) alongside the existing latest and
immutable X.Y.Z tags, so operators who want the latest patch on a given
minor line can pin without chasing every release. Since every 0.x minor
is potentially breaking, the {{major}}-only tag is deferred until 1.0.
cosign is installed via the sigstore/cosign-installer action pinned to
a full commit SHA. The signing step batches every tag produced by
metadata-action into a single cosign invocation pinned to the built
digest, so all tags resolve to the same manifest and we save N-1
Rekor/Fulcio round trips. The job declares id-token: write so the
ambient OIDC token can be minted.
A new docs page (docs/reference/verifying-images.mdx) walks users
through installing cosign, verifying an image, and inspecting the SBOM
and provenance with docker buildx imagetools. A note calls out that
signatures start shipping with v0.43.0, so older images failing
verification is expected behavior and not tampering.
Pin three third-party actions to full commit SHAs with version comments
so dependabot can update them in place without exposing the pipeline to
upstream tag tampering:
- aquasecurity/trivy-action@v0.35.0
- peter-evans/create-pull-request@v8.1.1
- googleapis/release-please-action@v4.4.0
Add a workflow-level permissions: contents: read default to ci.yml, with
update-screenshots keeping its explicit contents: write + pull-requests:
write override. Add the same default to docker-publish.yml (single job).
Expand .dockerignore to explicitly exclude .env*, *.key, *.pem, .github/,
docs/, website/, e2e/, tests/, test-results/, and playwright-report/ as
defense in depth against accidental build-context leaks. None of these
paths are read by any COPY step in Dockerfile.
The "Updating Sencho..." overlay used to dismiss prematurely while the
image pull was still running, after which the local node card would get
stuck in "updating" and eventually surface a generic "Timed Out" error
while the container remained on the old version.
Three root causes are addressed:
1. The image pull was synchronous (`execFileSync`), which blocked the
Node event loop. The overlay's health probe saw the server come back
the moment the pull finished and reloaded the page, even though the
container had not restarted yet. The pull is now async via
`promisify(execFile)`, so /api/health and /api/fleet/update-status
keep serving throughout.
2. The overlay reloaded on the first 200 from /api/health regardless of
whether the underlying process had actually restarted. /api/health
now exposes the gateway boot timestamp, and the overlay captures it
pre-update and only reloads when it observes a different value. A
wasOffline-then-online fallback handles the case where the pre-update
fetch failed.
3. Helper container spawn errors from `docker run` were silently
discarded, so a failed compose recreate never surfaced anywhere.
Errors are now captured into `lastUpdateError` via the execFile
callback and surfaced through the existing /api/fleet/update-status
error path.
A 3-minute early-fail heuristic on the local node block surfaces a clear
failure message when the helper fails silently, instead of waiting the
full 5-minute timeout for an unknown failure.
Replaces five ad-hoc in-process caches (project name map, templates, latest
version, fleet update status, remote node meta) with a single internal
CacheService that provides TTL, inflight-promise deduplication to protect
against thundering herd, stale-on-error fallback, and per-namespace
hit/miss/stale/size counters for observability.
Wraps the hot-path dashboard endpoints in the cache with write-path
invalidation: /api/stats (2s), /api/system/stats (3s), and
/api/stacks/statuses (3s). Keys are namespaced by nodeId so switching nodes
never serves another node's data. Every route that mutates container or
stack state calls invalidateNodeCaches(nodeId), which also drops the global
project-name-map, so user actions stay instantly reflected in the UI.
For /api/system/stats the cheap per-request network rx/tx block is kept
outside the cache so live-updating charts stay smooth while the expensive
systeminformation.currentLoad() CPU sample (~200ms) is reused across the
TTL.
Adds admin-only GET /api/system/cache-stats returning per-namespace
counters for operators who want to observe cache effectiveness.
Enables the compression middleware site-wide for JSON responses. Large
payloads like /api/templates shrink roughly 5x on the wire. SSE endpoints
are explicitly excluded via a Content-Type filter so live log tails and
metric streams are not buffered.
Bumps vitest hookTimeout to match testTimeout (15s) so parallel fork
workers do not hit the default 10s hook limit under CPU contention.
Adds 35 new tests (26 unit for CacheService, 9 integration for cached
endpoints) covering TTL expiry, inflight dedup, stale-on-error,
namespace invalidation, entry-cap safety guard, and write-path
invalidation end-to-end through Express routes.
* ci: optimize workflows to cut redundant runs and add timeout protection
Skip validation jobs (build, test, lint, Docker, E2E) on push-to-main
since code was already validated on the PR. Skip CI entirely for
release-please PRs (version bump only). Add timeout-minutes to all jobs
to prevent hung runners from burning 360 min (GitHub default). Add npm
caching to the start-app composite action. Add concurrency control to
docker-publish. Group Dependabot updates to batch PRs per ecosystem.
Estimated savings: ~10 min per release cycle, ~30-40 min/day.
* chore: add tsbuildinfo to gitignore
* fix(fleet): add Docker Hub fallback for version detection on private repos
The GitHub Releases API returns 404 for private repos, causing the
latest version fetch to silently fail and fall back to the gateway's
own version (defeating the update detection fix from PR #454).
Now tries GitHub first, then falls back to Docker Hub tags API which
is always public. Adds console.warn logging on fetch failures per
Directive 7.
* ci: trigger CI re-run
* fix(api): add tiered rate limiting to prevent polling lockouts
Replace the single global rate limiter (100 req/min/IP) with a tiered
system that separates high-frequency polling endpoints from standard
API traffic:
- Polling tier (300/min): /stats, /system/stats, /stacks/statuses,
/metrics/historical, /health, /meta, /auth/status, /auth/sso/providers,
/license. Exempt from the global limiter but governed by their own
safety net to prevent resource exhaustion.
- Standard tier (200/min): All other endpoints, raised from 100.
- Webhook tier (500/min): POST /webhooks/:id/trigger, dedicated limiter
for CI/CD platforms sharing datacenter IPs.
- Auth tier: Unchanged (5-10 attempts / 15 min).
Enterprise adaptations:
- Authenticated requests keyed by user session (JWT sub/username) instead
of IP, preventing shared NAT/VPN environments from pooling budgets.
- Internal node-to-node traffic (node_proxy tokens) bypasses all rate
limiters entirely.
Includes comprehensive stress tests (21 cases) validating tier
separation, node proxy bypass, and per-user keying.
* chore(deps): bump axios to 1.15.0 to fix SSRF vulnerability
Addresses GHSA-3p68-rc4w-qgx5 (NO_PROXY hostname normalization bypass).
Use a separate recheckingUpdates state for the Recheck button so the
modal content stays visible while refreshing. The full-screen loading
spinner (checkingUpdates) is now only used for the initial "Check Updates"
open from the header.
* fix(fleet): detect updates via GitHub Releases instead of gateway self-comparison
The fleet update check compared each node's version against the gateway's
own version, so the local node could never appear outdated. Now fetches
the actual latest release from GitHub Releases API with a 30-minute
in-memory cache and thundering-herd protection. The Recheck button
invalidates this cache via ?recheck=true to force a fresh lookup.
* docs(fleet): update docs to reflect GitHub Releases version detection
Replace "Gateway version" references with "Latest version" to match the
new label. Document that version comparison uses the latest GitHub release
rather than the gateway's own version, and that Recheck refreshes the
cached latest version.
Replace bg-card/80 with bg-popover/95 and match the backdrop-blur/saturate
values used by dialogs, dropdowns, and popovers. Remove per-type colored
gradient overlays so color differentiation comes from icons only. Unify the
determinate progress bar gradient with the loading bar's brand gradient.
sencho-token-app[bot]