Compare commits

..

11 Commits

Author SHA1 Message Date
Greg Bergé de2d468677 Fix build 2025-01-09 10:03:57 +01:00
Greg Bergé 7f50ad68bb Remove import 2025-01-09 09:49:51 +01:00
Greg Bergé 3a759b14a4 Fix lockfile 2025-01-09 09:46:55 +01:00
Greg Bergé b5e2e3fa8d Setup Sentry with Cloudflare 2025-01-09 09:40:47 +01:00
Greg Bergé fec48ce708 Import Sentry in middleware 2025-01-09 09:21:52 +01:00
Greg Bergé 0784bf5df7 Update wrangler 2025-01-09 08:49:48 +01:00
Greg Bergé f8f485ae25 Add log 2025-01-09 08:48:44 +01:00
Greg Bergé 3491bdc522 Import Sentry in middleware 2025-01-08 17:14:16 +01:00
Greg Bergé 2cda8d4cad Add more logs 2025-01-08 16:51:07 +01:00
Greg Bergé 3fcbe46262 Introduce error on purpose 2025-01-08 16:34:30 +01:00
Greg Bergé 884ee7073f Simplify Sentry setup, add logs 2025-01-08 16:34:27 +01:00
1104 changed files with 21372 additions and 158893 deletions
+1
View File
@@ -0,0 +1 @@
1.1.40
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix empty sitemap
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Adapt code to pull token from customer backend generated custom cookies
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Log component stack in Sentry
+5
View File
@@ -0,0 +1,5 @@
---
'@gitbook/react-openapi': minor
---
Add an optional client context to get a callback called when the Scalar client is opened for a block.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix pass Sentry release properly
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Add support for sidebar background styles
+7
View File
@@ -0,0 +1,7 @@
---
'gitbook': patch
---
Ignore invalid API calls to `getSiteRedirectFromSource` API
To reduce the load on the API and also avoid errors.
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
Fix the docs embed `navigateToPage` API on multi-space sites. Deep-linking to a page in a different space/section (e.g. `navigateToPage('/help-center/integrations')`) previously 404'd because the section base was not placed before `~gitbook/embed/page`. The target is now resolved to its space server-side, so pages in any space resolve correctly. The input accepts the page path, an absolute path, or the full published URL.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Ignore errors from event flushing
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Improve emoji setup, align with GitBook app
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Fix an issue where the redirects of potentially malicious images were not going through.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Make cookies access safer
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Improve contrast of search box placeholder
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Simplify the `useHash` algo used.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Track events for site insights using the new dedicated API.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Improve colour contrast of list item decoration
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Set Sentry release
+6
View File
@@ -0,0 +1,6 @@
---
'@gitbook/react-openapi': minor
'gitbook': minor
---
Revert scalar to 1.0.87 to mitigate an issue with ApiClientModalProvider
+6
View File
@@ -0,0 +1,6 @@
---
'@gitbook/react-contentkit': minor
'gitbook': minor
---
Send redirectOnError param to getPublishedContent when token is pulled from cookie
+5
View File
@@ -0,0 +1,5 @@
---
'@gitbook/react-contentkit': patch
---
Fix potential invalid URL error in react-contentkit
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Avoid error on fetch by passing a string URL
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Track clicks on links (header, footer, content) for site insights.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix code format overriding inline link styles
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix emojis display
+7
View File
@@ -0,0 +1,7 @@
---
'gitbook': patch
---
Implement a safer way to interact with localStorage.
If it's disabled on the browser it should not throw error.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix two issues where pages would crash due Recoil not behaving correctly in RSC.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix dynamic tabs infinite loop
+5
View File
@@ -0,0 +1,5 @@
---
'@gitbook/react-openapi': patch
---
Prevent codemirror from loading multiple versions in scalar
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
Sync the API reference responses selector with the "Responses" collapsibles, and keep the selected response in sync across every operation on the page (like the code sample language selector). Selecting a status code now expands the matching response section and applies to all operations at once.
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
Fix a light/dark flash on published sites configured to respect the system default (no theme toggle). Such sites forced the `system` theme, but `next-themes`' pre-paint script applies a forced value verbatim without resolving `prefers-color-scheme`, so the page painted light and only switched to dark after hydration. We now leave the theme unforced when the default is `system` (only concrete light/dark themes are forced), letting `next-themes`' existing pre-paint script resolve the system preference before first paint.
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
Separate the prompt block actions into a primary "Open in" dropdown and a secondary "Copy prompt" button, instead of a single combined button group, and align the block's design with the expandable block (bordered frame, left disclosure chevron, and subtle elevation when expanded).
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
Fix the code-block copy button not copying (and logging a console error) in the AI Assistant embed. The async Clipboard API can be unavailable or rejected in cross-origin iframe / non-secure embed contexts; we now fall back to a legacy `execCommand('copy')` write and only show the "copied" state when the copy actually succeeds.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix variant selector contrast for non-default themes
+5
View File
@@ -0,0 +1,5 @@
---
'@gitbook/react-openapi': patch
---
Updated scalar depdenency
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
Refine the per-paragraph AI ask button: shorten its tooltip to "Ask" (from "Ask <assistant> about this"), and hide it inside cards where it would otherwise be clipped by the card's overflow.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix search loading state
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix Sentry instrumentation
-5
View File
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
Keep the "On this page" and "Ask" buttons pinned below the header while scrolling on desktop API reference pages, so the page outline stays reachable throughout long operations.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Even safer localStorage
-5
View File
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
Hide unfocusable unlabelled button from screen readers
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Avoid errors on legacy browsers
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Improve readability of tables with hover style and vertical dividers
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Image optimization endpoint redirects to underlying image URL if the signature is not the latest.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix bullet list display on full size blocks
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix "Parser" language syntax highlighting
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Fix a vulnerability issue for images using an older version of the image signing parameter.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': minor
---
Track an event into site insights when visitor is opening the Scalar API client.
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Upgrade Next.js to v15, upgrade Shiki and use JS RegExp engine
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Improve smoothness of scroll listener
-5
View File
@@ -1,5 +0,0 @@
---
"gitbook": patch
---
a11y screen reader fixes
+5
View File
@@ -0,0 +1,5 @@
---
'gitbook': patch
---
Fix Sentry instrumentation server-side
-12
View File
@@ -1,12 +0,0 @@
{
"version": "0.0.1",
"configurations": [
{
"name": "🚀 Dev server",
"runtimeExecutable": "sh",
"runtimeArgs": ["-c", "bun install && bun dev"],
"port": 3000,
"autoPort": false
}
]
}
-3
View File
@@ -1,3 +0,0 @@
# Changes to the API data cache functions can invalidate all existing data cache
# causing a massive amount of revalidation, impacting our API.
packages/gitbook/src/lib/data/api.ts @SamyPesse
+6 -28
View File
@@ -53,41 +53,19 @@ After forking this repository, you'll want to [create a branch](https://docs.git
#### 3. Install dependencies and run the project locally
##### Prerequisites:
GitBook uses [Bun](https://bun.sh/) to run the project. Make sure you're using the specified version of `node` before running any of the development commands to ensure a smooth development experience.
- [Node.js](https://nodejs.org/en) (see "engines" in `package.json`)
- [Bun](https://bun.sh/) (see "packageManager" in `package.json`)
You can easily do this by running the command `nvm use`.
##### Setup steps:
1. Ensure you are using the project's version of Node:
```bash
nvm use
```
2. Install dependencies using Bun:
```bash
bun install
```
3. Start the development server:
```bash
bun dev
```
Additional development commands:
- `bun format`: Format the code using Biome
- `bun typecheck`: Run TypeScript type checking
- `bun unit`: Run unit tests
- `bun e2e`: Run end-to-end tests
To start your local version of GitBook, run the command `bun dev`.
#### 4. Preview your changes
When running the development server, published GitBook sites can be rendered through your local version at `http://localhost:3000/url`.
When running the development server, published GitBook sites can be rendered through your local version at `http://localhost:3000/`.
For example, our published docs can be viewed using the local version by visiting `http://localhost:3000/url/gitbook.com/docs` after running the development server.
For example, our published docs can be viewed using the local version by visiting `http://localhost:3000/docs.gitbook.com` after running the development server.
You can visit any published GitBook site behind your development server. Please make sure your site is [published publicly](https://gitbook.com/docs/published-documentation/publish-your-content-as-a-docs-site) to ensure you can view the site correctly in your development version.
You can visit any published GitBook site behind your development server. Please make sure your site is [published publicly](https://docs.gitbook.com/published-documentation/publish-your-content-as-a-docs-site) to ensure you can view the site correctly in your development version.
### Commit your update
@@ -1,83 +0,0 @@
name: Gradual Deploy to Cloudflare
description: Use gradual deployment to deploy to Cloudflare. This action will upload the middleware and server versions to Cloudflare and kept them bound together
inputs:
apiToken:
description: 'Cloudflare API token'
required: true
accountId:
description: 'Cloudflare account ID'
required: true
environment:
description: 'Cloudflare environment to deploy to (staging, production, preview)'
required: true
middlewareVersionId:
description: 'Middleware version ID to deploy'
required: true
serverVersionId:
description: 'Server version ID to deploy'
required: true
outputs:
deployment-url:
description: "Deployment URL"
value: ${{ steps.deploy_middleware.outputs.deployment-url }}
runs:
using: 'composite'
steps:
- id: wrangler_status
name: Check wrangler deployment status
uses: cloudflare/wrangler-action@v3.14.0
with:
apiToken: ${{ inputs.apiToken }}
accountId: ${{ inputs.accountId }}
workingDirectory: ./
wranglerVersion: '4.43.0'
environment: ${{ inputs.environment }}
command: deployments status --config ./packages/gitbook/openNext/customWorkers/defaultWrangler.jsonc
# This step is used to get the version ID that is currently deployed to Cloudflare.
- id: extract_current_version
name: Extract current version
shell: bash
run: |
version_id=$(echo "${{ steps.wrangler_status.outputs.command-output }}" | grep -A 3 "(100%)" | grep -oP '[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}')
echo "version_id=$version_id" >> $GITHUB_OUTPUT
- id: deploy_server
name: Deploy server to Cloudflare at 0%
uses: cloudflare/wrangler-action@v3.14.0
with:
apiToken: ${{ inputs.apiToken }}
accountId: ${{ inputs.accountId }}
workingDirectory: ./
wranglerVersion: '4.43.0'
environment: ${{ inputs.environment }}
command: versions deploy ${{ steps.extract_current_version.outputs.version_id }}@100% ${{ inputs.serverVersionId }}@0% -y --config ./packages/gitbook/openNext/customWorkers/defaultWrangler.jsonc
# Since we use version overrides headers, we can directly deploy the middleware to 100%.
- id: deploy_middleware
name: Deploy middleware to Cloudflare at 100%
uses: cloudflare/wrangler-action@v3.14.0
with:
apiToken: ${{ inputs.apiToken }}
accountId: ${{ inputs.accountId }}
workingDirectory: ./
wranglerVersion: '4.43.0'
environment: ${{ inputs.environment }}
command: versions deploy ${{ inputs.middlewareVersionId }}@100% -y --config ./packages/gitbook/openNext/customWorkers/middlewareWrangler.jsonc
- name: Deploy server to Cloudflare at 100%
uses: cloudflare/wrangler-action@v3.14.0
with:
apiToken: ${{ inputs.apiToken }}
accountId: ${{ inputs.accountId }}
workingDirectory: ./
wranglerVersion: '4.43.0'
environment: ${{ inputs.environment }}
command: versions deploy ${{ inputs.serverVersionId }}@100% -y --config ./packages/gitbook/openNext/customWorkers/defaultWrangler.jsonc
- name: Outputs
shell: bash
env:
DEPLOYMENT_URL: ${{ steps.deploy_middleware.outputs.deployment-url }}
run: |
echo "URL: ${{ steps.deploy_middleware.outputs.deployment-url }}"
@@ -0,0 +1,37 @@
name: 'Setup Playwright'
description: 'Install Playwright and dependencies'
runs:
using: 'composite'
steps:
# Run npm ci and get Playwright version
- name: 🏗 Prepare Playwright env
shell: bash
run: |
PLAYWRIGHT_VERSION=$(npm ls --json @playwright/test | jq --raw-output '.dependencies["gitbook"].dependencies["@playwright/test"].version')
echo "PLAYWRIGHT_VERSION=$PLAYWRIGHT_VERSION" >> $GITHUB_ENV
# Cache browser binaries, cache key is based on Playwright version and OS
- name: 🧰 Cache Playwright browser binaries
uses: actions/cache@v4
id: playwright-cache
with:
path: '~/.cache/ms-playwright'
key: '${{ runner.os }}-playwright-${{ env.PLAYWRIGHT_VERSION }}'
restore-keys: |
${{ runner.os }}-playwright-
# Install browser binaries & OS dependencies if cache missed
- name: 🏗 Install Playwright browser binaries & OS dependencies
if: steps.playwright-cache.outputs.cache-hit != 'true'
shell: bash
working-directory: packages/gitbook
run: |
bun x playwright install --with-deps chromium
# Install only the OS dependencies if cache hit
- name: 🏗 Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
shell: bash
working-directory: packages/gitbook
run: |
bun x playwright install-deps
@@ -1,144 +0,0 @@
name: 'Deploy cloudflare'
description: 'Deploy GitBook to Cloudflare'
inputs:
opItem:
description: '1Password item to load secrets from'
required: true
opServiceAccount:
description: '1Password service account token'
required: true
apiToken:
description: 'Cloudflare API token'
required: true
accountId:
description: 'Cloudflare account ID'
required: true
environment:
description: 'Cloudflare environment to deploy to (staging, production, preview)'
required: true
deploy:
description: 'Deploy as main version for all traffic instead of uploading versions'
required: true
commitTag:
description: 'Commit branch to associate with the deployment'
required: true
commitMessage:
description: 'Commit message to associate with the deployment'
required: true
outputs:
deployment-url:
description: "Deployment URL"
value: ${{ steps.upload_middleware.outputs.deployment-url }}
runs:
using: 'composite'
steps:
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
shell: bash
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Load secret
uses: 1password/load-secrets-action@v2
env:
OP_SERVICE_ACCOUNT_TOKEN: ${{ inputs.opServiceAccount }}
GITBOOK_URL: ${{ inputs.opItem }}/GITBOOK_URL
GITBOOK_ICONS_URL: ${{ inputs.opItem }}/GITBOOK_ICONS_URL
GITBOOK_ICONS_TOKEN: ${{ inputs.opItem }}/GITBOOK_ICONS_TOKEN
NEXT_SERVER_ACTIONS_ENCRYPTION_KEY: ${{ inputs.opItem }}/NEXT_SERVER_ACTIONS_ENCRYPTION_KEY
GITBOOK_SECRET: ${{ inputs.opItem }}/GITBOOK_SECRET
GITBOOK_APP_URL: ${{ inputs.opItem }}/GITBOOK_APP_URL
GITBOOK_API_URL: ${{ inputs.opItem }}/GITBOOK_API_URL
GITBOOK_API_PUBLIC_URL: ${{ inputs.opItem }}/GITBOOK_API_PUBLIC_URL
GITBOOK_API_TOKEN: ${{ inputs.opItem }}/GITBOOK_API_TOKEN
GITBOOK_OAUTH_SERVER_URL: ${{ inputs.opItem }}/GITBOOK_OAUTH_SERVER_URL
GITBOOK_PREVIEW_BASE_URL: ${{ inputs.opItem }}/GITBOOK_PREVIEW_BASE_URL
GITBOOK_INTEGRATIONS_HOST: ${{ inputs.opItem }}/GITBOOK_INTEGRATIONS_HOST
GITBOOK_INTEGRATIONS_CONTENT_HOST: ${{ inputs.opItem }}/GITBOOK_INTEGRATIONS_CONTENT_HOST
GITBOOK_IMAGE_RESIZE_SIGNING_KEY: ${{ inputs.opItem }}/GITBOOK_IMAGE_RESIZE_SIGNING_KEY
GITBOOK_IMAGE_RESIZE_URL: ${{ inputs.opItem }}/GITBOOK_IMAGE_RESIZE_URL_2
GITBOOK_IMAGE_RESIZE_SALT: ${{ inputs.opItem }}/GITBOOK_IMAGE_RESIZE_SALT
GITBOOK_IMAGE_RESIZE_MODE: ${{ inputs.opItem }}/GITBOOK_IMAGE_RESIZE_MODE
GITBOOK_ASSETS_PREFIX: ${{ inputs.opItem }}/GITBOOK_ASSETS_PREFIX
GITBOOK_FONTS_URL: ${{ inputs.opItem }}/GITBOOK_FONTS_URL
- name: Build worker
run: bun run turbo build:cloudflare
env:
GITBOOK_RUNTIME: cloudflare
GITBOOK_BLOCK_SEARCH_INDEXATION: ${{ inputs.environment == 'preview' && 'true' || '' }}
GITBOOK_ALLOW_CUSTOMIZATION_OVERRIDE: ${{ inputs.environment == 'preview' && 'true' || '' }}
shell: bash
- name: Upload the DO worker
uses: cloudflare/wrangler-action@v3.14.0
with:
apiToken: ${{ inputs.apiToken }}
accountId: ${{ inputs.accountId }}
workingDirectory: ./
wranglerVersion: '4.43.0'
environment: ${{ inputs.environment }}
command: deploy --config ./packages/gitbook/openNext/customWorkers/doWrangler.jsonc
- id: upload_server
name: Upload server to Cloudflare
uses: cloudflare/wrangler-action@v3.14.0
with:
apiToken: ${{ inputs.apiToken }}
accountId: ${{ inputs.accountId }}
workingDirectory: ./
wranglerVersion: '4.43.0'
environment: ${{ inputs.environment }}
command: ${{ format('versions upload --tag {0} --message "{1}"', inputs.commitTag, inputs.commitMessage) }} --config ./packages/gitbook/openNext/customWorkers/defaultWrangler.jsonc
- name: Extract server version worker ID
shell: bash
id: extract_server_version_id
run: |
version_id=$(echo '${{ steps.upload_server.outputs.command-output }}' | grep "Worker Version ID" | awk '{print $4}')
echo "version_id=$version_id" >> $GITHUB_OUTPUT
- name: Run updateWrangler scripts
shell: bash
run: |
bun run ./packages/gitbook/openNext/customWorkers/script/updateWrangler.ts ${{ steps.extract_server_version_id.outputs.version_id }}
- id: upload_middleware
name: Upload middleware to Cloudflare
uses: cloudflare/wrangler-action@v3.14.0
with:
apiToken: ${{ inputs.apiToken }}
accountId: ${{ inputs.accountId }}
workingDirectory: ./
wranglerVersion: '4.43.0'
environment: ${{ inputs.environment }}
command: ${{ format('versions upload --tag {0} --message "{1}"', inputs.commitTag, inputs.commitMessage) }} --config ./packages/gitbook/openNext/customWorkers/middlewareWrangler.jsonc
- name: Extract middleware version worker ID
shell: bash
id: extract_middleware_version_id
run: |
version_id=$(echo '${{ steps.upload_middleware.outputs.command-output }}' | grep "Worker Version ID" | awk '{print $4}')
echo "version_id=$version_id" >> $GITHUB_OUTPUT
- name: Deploy server and middleware to Cloudflare
if: ${{ inputs.deploy == 'true' }}
uses: ./.github/actions/gradual-deploy-cloudflare
with:
apiToken: ${{ inputs.apiToken }}
accountId: ${{ inputs.accountId }}
opServiceAccount: ${{ inputs.opServiceAccount }}
opItem: ${{ inputs.opItem }}
environment: ${{ inputs.environment }}
serverVersionId: ${{ steps.extract_server_version_id.outputs.version_id }}
middlewareVersionId: ${{ steps.extract_middleware_version_id.outputs.version_id }}
deploy: ${{ inputs.deploy }}
- name: Outputs
shell: bash
env:
DEPLOYMENT_URL: ${{ steps.upload_middleware.outputs.deployment-url }}
run: |
echo "URL: ${{ steps.upload_middleware.outputs.deployment-url }}"
echo "Output server: ${{ steps.upload_server.outputs.command-output }}"
-100
View File
@@ -1,100 +0,0 @@
name: 'Deploy vercel'
description: 'Deploy GitBook to Vercel'
inputs:
vercelOrg:
description: 'Vercel organization'
required: true
vercelProject:
description: 'Vercel project'
required: true
vercelToken:
description: 'Vercel token'
required: true
opItem:
description: '1Password item to load secrets from'
required: true
opServiceAccount:
description: '1Password service account token'
required: true
environment:
description: 'Environment to deploy to'
required: true
headSha:
description: 'Git ref to deploy, used for the deploymentId'
required: false
outputs:
deployment-url:
description: "Deployment URL"
value: ${{ steps.deploy.outputs.deployment-url }}
runs:
using: 'composite'
steps:
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
shell: bash
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Pull Vercel Environment Information
run: bun run vercel pull --yes --environment=${{ inputs.environment }} --token=${{ inputs.vercelToken }}
shell: bash
env:
VERCEL_ORG_ID: ${{ inputs.vercelOrg }}
VERCEL_PROJECT_ID: ${{ inputs.vercelProject }}
- name: Load secret
uses: 1password/load-secrets-action@v2
env:
OP_SERVICE_ACCOUNT_TOKEN: ${{ inputs.opServiceAccount }}
GITBOOK_URL: ${{ inputs.opItem }}/GITBOOK_URL
GITBOOK_ICONS_URL: ${{ inputs.opItem }}/GITBOOK_ICONS_URL
GITBOOK_ICONS_TOKEN: ${{ inputs.opItem }}/GITBOOK_ICONS_TOKEN
GITBOOK_SECRET: ${{ inputs.opItem }}/GITBOOK_SECRET
GITBOOK_APP_URL: ${{ inputs.opItem }}/GITBOOK_APP_URL
GITBOOK_API_URL: ${{ inputs.opItem }}/GITBOOK_API_URL
GITBOOK_API_PUBLIC_URL: ${{ inputs.opItem }}/GITBOOK_API_PUBLIC_URL
GITBOOK_API_TOKEN: ${{ inputs.opItem }}/GITBOOK_API_TOKEN
GITBOOK_OAUTH_SERVER_URL: ${{ inputs.opItem }}/GITBOOK_OAUTH_SERVER_URL
GITBOOK_PREVIEW_BASE_URL: ${{ inputs.opItem }}/GITBOOK_PREVIEW_BASE_URL
GITBOOK_INTEGRATIONS_HOST: ${{ inputs.opItem }}/GITBOOK_INTEGRATIONS_HOST
GITBOOK_INTEGRATIONS_CONTENT_HOST: ${{ inputs.opItem }}/GITBOOK_INTEGRATIONS_CONTENT_HOST
GITBOOK_IMAGE_RESIZE_SIGNING_KEY: ${{ inputs.opItem }}/GITBOOK_IMAGE_RESIZE_SIGNING_KEY
GITBOOK_IMAGE_RESIZE_URL: ${{ inputs.opItem }}/GITBOOK_IMAGE_RESIZE_URL_2
GITBOOK_IMAGE_RESIZE_SALT: ${{ inputs.opItem }}/GITBOOK_IMAGE_RESIZE_SALT
GITBOOK_IMAGE_RESIZE_MODE: ${{ inputs.opItem }}/GITBOOK_IMAGE_RESIZE_MODE
GITBOOK_ASSETS_PREFIX: ${{ inputs.opItem }}/GITBOOK_ASSETS_PREFIX
GITBOOK_FONTS_URL: ${{ inputs.opItem }}/GITBOOK_FONTS_URL
- name: Inject build env vars
if: ${{ inputs.environment == 'preview' }}
shell: bash
run: |
HEAD_SHA=$(git rev-parse HEAD)
echo "resolved HEAD_SHA: $HEAD_SHA"
echo "GITBOOK_HEAD_SHA=$HEAD_SHA" >> .vercel/.env.${{ inputs.environment }}.local
echo "GITBOOK_RUNTIME=vercel" >> .vercel/.env.${{ inputs.environment }}.local
echo "GITBOOK_BLOCK_SEARCH_INDEXATION=true" >> .vercel/.env.${{ inputs.environment }}.local
echo "GITBOOK_ALLOW_CUSTOMIZATION_OVERRIDE=true" >> .vercel/.env.${{ inputs.environment }}.local
echo "--- .vercel/.env.${{ inputs.environment }}.local after inject ---"
cat .vercel/.env.${{ inputs.environment }}.local
- name: Build Project Artifacts
run: bun run vercel build --target=${{ inputs.environment }} --token=${{ inputs.vercelToken }}
shell: bash
env:
VERCEL_ORG_ID: ${{ inputs.vercelOrg }}
VERCEL_PROJECT_ID: ${{ inputs.vercelProject }}
GITBOOK_RUNTIME: vercel
GITBOOK_HEAD_SHA: ${{ inputs.headSha }}
- name: Deploy Project Artifacts to Vercel
id: deploy
shell: bash
run: |
DEPLOYMENT_URL=$(bun run vercel deploy --prebuilt --target=${{ inputs.environment }} --token=${{ inputs.vercelToken }})
echo "deployment-url=$DEPLOYMENT_URL" >> "$GITHUB_OUTPUT"
env:
VERCEL_ORG_ID: ${{ inputs.vercelOrg }}
VERCEL_PROJECT_ID: ${{ inputs.vercelProject }}
- name: Outputs
shell: bash
run: |
echo "URL: ${{ steps.deploy.outputs.deployment-url }}"
+10 -1
View File
@@ -3,7 +3,16 @@ description: 'Install Bun and cache dependencies'
runs:
using: 'composite'
steps:
# We cache dependencies even if Bun is fast to avoid downloading fontawesome too many times
# and impacting bandwidth there.
- name: Cache bun
uses: actions/cache@v4
with:
path: ~/.bun/install/cache
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lockb') }}-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Setup bun
uses: oven-sh/setup-bun@v2
with:
bun-version-file: 'package.json'
bun-version-file: '.bun-version'
+107 -7
View File
@@ -4,12 +4,99 @@ on:
push:
branches:
- main
- changeset-release/main
env:
NPM_TOKEN_READONLY: ${{ secrets.NPM_TOKEN_READONLY }}
jobs:
deploy:
name: Deploy to Cloudflare Pages
runs-on: ubuntu-latest
permissions:
contents: read
deployments: write
issues: write
pull-requests: write
checks: write
statuses: write
outputs:
deployment_url: ${{ steps.deploy.outputs.deployment-url }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Sets env vars for production
run: |
echo "SENTRY_ENVIRONMENT=production" >> $GITHUB_ENV
echo "GITBOOK_ASSETS_PREFIX=https://static.gitbook.com" >> $GITHUB_ENV
if: startsWith(github.ref, 'refs/heads/main')
- name: Sets env vars for preview
run: |
echo "SENTRY_ENVIRONMENT=preview" >> $GITHUB_ENV
if: 1 && !startsWith(github.ref, 'refs/heads/main')
- name: Build Next.js with next-on-pages
run: bun run build:cloudflare
env:
SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}
SENTRY_ORG: ${{ vars.SENTRY_ORG }}
SENTRY_PROJECT: ${{ vars.SENTRY_PROJECT }}
SENTRY_DSN: ${{ vars.SENTRY_DSN }}
SENTRY_RELEASE: ${{ github.sha }}
- id: deploy
name: Deploy to Cloudflare
uses: cloudflare/wrangler-action@v3.11.0
with:
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
workingDirectory: ./
wranglerVersion: '3.82.0'
command: pages deploy ./packages/gitbook/.vercel/output/static --project-name=${{ vars.CLOUDFLARE_PROJECT_NAME }} --branch=${{ github.ref == 'refs/heads/main' && 'main' || format('pr{0}', github.event.pull_request.number) }}
- name: Outputs
run: |
echo "URL: ${{ steps.deploy.outputs.deployment-url }}"
echo "Alias URL: ${{ steps.deploy.outputs.deployment-alias-url }}"
- name: Archive build output
uses: actions/upload-artifact@v4
with:
name: build-output
path: .vercel/
# Until https://github.com/cloudflare/wrangler-action/issues/301 is done
- name: Update Deployment Status to Success
env:
DEPLOYMENT_URL: ${{ steps.deploy.outputs.deployment-url }}
run: |
curl -X POST \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
-H "Accept: application/vnd.github.v3+json" \
-d '{"state": "success", "target_url": "${{ steps.deploy.outputs.deployment-url }}", "description": "Deployed Preview URL for commit", "context": "cloudflare/preview"}' \
https://api.github.com/repos/${{ github.repository }}/statuses/${{ github.sha }}
- name: Find GitHub Comment
uses: peter-evans/find-comment@v3
id: fc
if: 1 && !startsWith(github.ref, 'refs/heads/main')
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: GitBook Preview
- name: Create or update GitHub comment
uses: peter-evans/create-or-update-comment@v4
if: 1 && !startsWith(github.ref, 'refs/heads/main')
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body: |
**GitBook Preview**
Latest commit: [${{ steps.deploy.outputs.deployment-url }}](${{ steps.deploy.outputs.deployment-url }})
PR: [${{ steps.deploy.outputs.deployment-alias-url }}](${{ steps.deploy.outputs.deployment-alias-url }})
edit-mode: replace
format:
runs-on: ubuntu-latest
name: Format
timeout-minutes: 6
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -20,10 +107,22 @@ jobs:
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- run: bun format:check
lint:
runs-on: ubuntu-latest
name: Lint
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- run: bun lint --no-cache
test:
runs-on: ubuntu-latest
name: Test
timeout-minutes: 6
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -34,10 +133,12 @@ jobs:
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- run: bun unit
build:
build-oss:
# CI to check that the repository builds correctly on a machine without the credentials
runs-on: ubuntu-latest
name: Build
timeout-minutes: 6
name: Build (Open Source)
env:
NPM_TOKEN_READONLY: ''
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -51,7 +152,6 @@ jobs:
typecheck:
runs-on: ubuntu-latest
name: Typecheck
timeout-minutes: 6
steps:
- name: Checkout
uses: actions/checkout@v4
-252
View File
@@ -1,252 +0,0 @@
name: Preview
on:
pull_request_target:
push:
branches:
- main
jobs:
approval:
# Ensure that external contributors' PRs are manually approved before deployment
name: Approve Preview Deployment
runs-on: ubuntu-latest
if: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.fork }}
environment:
name: preview-approval
steps:
- name: Approval gate
run: echo "Preview deployment approved."
deploy-v2-vercel:
name: Deploy v2 to Vercel (preview)
runs-on: ubuntu-latest
needs: approval
if: ${{ always() && (needs.approval.result == 'success' || needs.approval.result == 'skipped') }}
environment:
name: 2v-preview
url: ${{ steps.deploy.outputs.deployment-url }}
outputs:
deployment-url: ${{ steps.deploy.outputs.deployment-url }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Deploy to Vercel
id: deploy
uses: ./.github/composite/deploy-vercel
with:
environment: preview
vercelOrg: ${{ secrets.VERCEL_ORG_ID }}
vercelProject: ${{ secrets.VERCEL_PROJECT_ID }}
vercelToken: ${{ secrets.VERCEL_TOKEN }}
opItem: op://gitbook-open/2v-preview
opServiceAccount: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
headSha: ${{ github.event.pull_request.head.sha || github.sha }}
deploy-v2-cloudflare:
name: Deploy v2 to Cloudflare Worker (preview)
runs-on: ubuntu-latest
needs: approval
if: ${{ always() && (needs.approval.result == 'success' || needs.approval.result == 'skipped') }}
environment:
name: 2c-preview
url: ${{ steps.deploy.outputs.deployment-url }}
outputs:
deployment-url: ${{ steps.deploy.outputs.deployment-url || steps.extract-worker-id.outputs.worker-url }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Deploy to Cloudflare
id: deploy
uses: ./.github/composite/deploy-cloudflare
with:
environment: preview
deploy: ${{ github.ref == 'refs/heads/main' }}
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
opItem: op://gitbook-open/2c-preview
opServiceAccount: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
commitTag: ${{ github.ref == 'refs/heads/main' && 'main' || format('pr{0}', github.event.pull_request.number) }}
commitMessage: ${{ github.sha }}
- name: Extract Worker ID
id: extract-worker-id
if: ${{ !steps.deploy.outputs.deployment-url }}
run: |
if [[ "${{ steps.deploy.outputs.command-output }}" =~ Worker\ Version\ ID:\ ([0-9a-f]{8})-([0-9a-f-]+) ]]; then
WORKER_ID_FIRST_PART="${BASH_REMATCH[1]}"
echo "worker-url=https://${WORKER_ID_FIRST_PART}-gitbook-open-v2-preview.gitbook.workers.dev/" >> $GITHUB_OUTPUT
fi
- name: Outputs
run: |
echo "URL: ${{ steps.deploy.outputs.deployment-url || steps.extract-worker-id.outputs.worker-url }}"
comment-deployments:
runs-on: ubuntu-latest
name: Comment Deployments (preview)
if: always() && github.event_name != 'push'
needs:
- deploy-v2-vercel
- deploy-v2-cloudflare
steps:
- name: Find GitHub Comment
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: 'Summary of the deployments'
- name: Create or update GitHub comment
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body: |
Summary of the deployments:
| Version | URL | Status |
| --- | --- | --- |
| Vercel | [${{ needs.deploy-v2-vercel.outputs.deployment-url }}](${{ needs.deploy-v2-vercel.outputs.deployment-url }}) | ${{ needs.deploy-v2-vercel.result == 'success' && '✅' || '❌' }} |
| Cloudflare | [${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}](${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}) | ${{ needs.deploy-v2-cloudflare.result == 'success' && '✅' || '❌' }} |
### Test content
| Site | `2v` | `2c` |
| --- | --- | --- |
| GitBook | [${{ needs.deploy-v2-vercel.outputs.deployment-url }}/url/gitbook.com/docs](${{ needs.deploy-v2-vercel.outputs.deployment-url }}/url/gitbook.com/docs) | [${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}/url/gitbook.com/docs](${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}/url/gitbook.com/docs) |
| E2E | [${{ needs.deploy-v2-vercel.outputs.deployment-url }}/url/gitbook.gitbook.io/test-gitbook-open](${{ needs.deploy-v2-vercel.outputs.deployment-url }}/url/gitbook.gitbook.io/test-gitbook-open) | [${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}/url/gitbook.gitbook.io/test-gitbook-open](${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}/url/gitbook.gitbook.io/test-gitbook-open) |
edit-mode: replace
visual-testing-v2-vercel:
runs-on: ubuntu-latest
name: Visual Testing v2
needs: deploy-v2-vercel
if: ${{ always() && needs.deploy-v2-vercel.result == 'success' }}
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Run Playwright tests
run: bun e2e
env:
BASE_URL: ${{ needs.deploy-v2-vercel.outputs.deployment-url }}
SITE_BASE_URL: ${{ needs.deploy-v2-vercel.outputs.deployment-url }}/url/
ARGOS_TOKEN: ${{ secrets.ARGOS_TOKEN }}
ARGOS_BUILD_NAME: v2-vercel
visual-testing-v2-cloudflare:
runs-on: ubuntu-latest
name: Visual Testing v2 (Cloudflare)
needs: deploy-v2-cloudflare
if: ${{ always() && needs.deploy-v2-cloudflare.result == 'success' }}
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Run Playwright tests
run: bun e2e
env:
BASE_URL: ${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}
SITE_BASE_URL: ${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}/url/
ARGOS_TOKEN: ${{ secrets.ARGOS_TOKEN }}
ARGOS_BUILD_NAME: v2-cloudflare
visual-testing-customers-v2:
runs-on: ubuntu-latest
name: Visual Testing Customers v2
needs: deploy-v2-vercel
if: ${{ always() && needs.deploy-v2-vercel.result == 'success' }}
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Run Playwright tests
run: bun e2e-customers
env:
BASE_URL: ${{ needs.deploy-v2-vercel.outputs.deployment-url }}
SITE_BASE_URL: ${{ needs.deploy-v2-vercel.outputs.deployment-url }}/url/
ARGOS_TOKEN: ${{ secrets.ARGOS_TOKEN }}
ARGOS_BUILD_NAME: customers-v2-vercel
visual-testing-customers-v2-cloudflare:
runs-on: ubuntu-latest
name: Visual Testing Customers v2 (Cloudflare)
needs: deploy-v2-cloudflare
if: ${{ always() && needs.deploy-v2-cloudflare.result == 'success' }}
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Run Playwright tests
run: bun e2e-customers
env:
BASE_URL: ${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}
SITE_BASE_URL: ${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}/url/
ARGOS_TOKEN: ${{ secrets.ARGOS_TOKEN }}
ARGOS_BUILD_NAME: customers-v2-cloudflare
browserless-testing-v2-vercel:
runs-on: ubuntu-latest
name: Browserless Testing v2 (Vercel)
needs: deploy-v2-vercel
if: ${{ always() && needs.deploy-v2-vercel.result == 'success' }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Run tests
run: cd ./packages/gitbook && bun e2e-browserless
env:
BASE_URL: ${{needs.deploy-v2-vercel.outputs.deployment-url}}
SITE_BASE_URL: ${{ needs.deploy-v2-vercel.outputs.deployment-url }}/url/
# browserless-testing-v2-cloudflare:
# runs-on: ubuntu-latest
# name: Browserless Testing v2 (Cloudflare)
# needs: deploy-v2-cloudflare
# steps:
# - name: Checkout
# uses: actions/checkout@v4
# - name: Setup Bun
# uses: ./.github/composite/setup-bun
# - name: Install dependencies
# run: bun install --frozen-lockfile
# env:
# PUPPETEER_SKIP_DOWNLOAD: 1
# - name: Run tests
# run: cd ./packages/gitbook && bun e2e-browserless
# env:
# BASE_URL: ${{needs.deploy-v2-cloudflare.outputs.deployment-url}}
# SITE_BASE_URL: ${{ needs.deploy-v2-cloudflare.outputs.deployment-url }}/url/
-53
View File
@@ -1,53 +0,0 @@
name: Production
on:
push:
branches:
- main
jobs:
deploy-v2-vercel:
name: Deploy v2 to Vercel (production)
runs-on: ubuntu-latest
environment:
name: 2v-production
url: ${{ steps.deploy.outputs.deployment-url }}
outputs:
deployment-url: ${{ steps.deploy.outputs.deployment-url }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Deploy
id: deploy
uses: ./.github/composite/deploy-vercel
with:
environment: production
vercelOrg: ${{ secrets.VERCEL_ORG_ID }}
vercelProject: ${{ secrets.VERCEL_PROJECT_ID }}
vercelToken: ${{ secrets.VERCEL_TOKEN }}
opItem: op://gitbook-open/2v-production
opServiceAccount: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
deploy-v2-cloudflare:
name: Deploy v2 to Cloudflare Worker (production)
runs-on: ubuntu-latest
environment:
name: 2c-production
url: ${{ steps.deploy.outputs.deployment-url }}
outputs:
deployment-url: ${{ steps.deploy.outputs.deployment-url }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Deploy
id: deploy
uses: ./.github/composite/deploy-cloudflare
with:
environment: production
deploy: true
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
opItem: op://gitbook-open/2c-production
opServiceAccount: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
commitTag: main
commitMessage: ${{ github.sha }}
- name: Outputs
run: |
echo "URL: ${{ steps.deploy.outputs.deployment-url }}"
-53
View File
@@ -1,53 +0,0 @@
name: Staging
on:
push:
branches:
- main
jobs:
deploy-v2-vercel:
name: Deploy v2 to Vercel (staging)
runs-on: ubuntu-latest
environment:
name: 2v-staging
url: ${{ steps.deploy.outputs.deployment-url }}
outputs:
deployment-url: ${{ steps.deploy.outputs.deployment-url }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Deploy
id: deploy
uses: ./.github/composite/deploy-vercel
with:
environment: staging
vercelOrg: ${{ secrets.VERCEL_ORG_ID }}
vercelProject: ${{ secrets.VERCEL_PROJECT_ID }}
vercelToken: ${{ secrets.VERCEL_TOKEN }}
opItem: op://gitbook-open/2v-staging
opServiceAccount: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
deploy-v2-cloudflare:
name: Deploy v2 to Cloudflare Worker (staging)
runs-on: ubuntu-latest
environment:
name: 2c-staging
url: ${{ steps.deploy.outputs.deployment-url }}
outputs:
deployment-url: ${{ steps.deploy.outputs.deployment-url }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Deploy
id: deploy
uses: ./.github/composite/deploy-cloudflare
with:
environment: staging
deploy: true
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
opItem: op://gitbook-open/2c-staging
opServiceAccount: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
commitTag: main
commitMessage: ${{ github.sha }}
- name: Outputs
run: |
echo "URL: ${{ steps.deploy.outputs.deployment-url }}"
+25 -21
View File
@@ -4,49 +4,53 @@ on:
push:
branches:
- main
concurrency: ${{ github.workflow }}-${{ github.ref }}
jobs:
publish:
name: Publish
runs-on: ubuntu-latest
permissions:
id-token: write
pull-requests: write
contents: write
env:
# The repo is managed by Bun, but publishing uses npm for provenance.
# Keep Corepack from rejecting packageManager: bun@... when npm runs.
COREPACK_ENABLE_PROJECT_SPEC: 0
steps:
- name: Checkout Repo
uses: actions/checkout@v3
with:
# This makes Actions fetch all Git history so that Changesets can generate changelogs with the correct commits
fetch-depth: 0
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: 22
registry-url: 'https://registry.npmjs.org'
- name: Update npm
run: |
corepack enable npm
corepack install -g npm@latest
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Create Release Pull Request
- name: Create Release Pull Request or Publish to npm
id: changesets
uses: changesets/action@v1
with:
publish: bun run publish-all-packages
version: bun run changeset-version
publish: npm run release
env:
# Using a PAT instead of GITHUB_TOKEN because we need to run workflows when releases are created
# https://github.com/orgs/community/discussions/26875#discussioncomment-3253761
GITHUB_TOKEN: ${{ secrets.GH_PERSONAL_TOKEN }}
NPM_TOKEN_READONLY: ${{ secrets.NPM_TOKEN_READONLY }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
release-preview:
# For now it releases the cache-do to both preview and production
# Once we changed to deploy the app only on release, we should change `release:preview` in `cache-do`
name: Release Preview
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout@v3
- name: Setup Bun
uses: ./.github/composite/setup-bun
- name: Install dependencies
run: bun install --frozen-lockfile
env:
PUPPETEER_SKIP_DOWNLOAD: 1
- name: Release preview packages
run: bun run release:preview
env:
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
-7
View File
@@ -22,10 +22,3 @@ yarn-error.log*
# Env files
.env.local
# TypeScript
*.tsbuildinfo
.npmrc
# Bun pack artifacts
packages/*/*.tgz
+1 -1
View File
@@ -1 +1 @@
v22.3
v20.6
+10
View File
@@ -0,0 +1,10 @@
.next
.vercel
# Generated
packages/emoji-codepoints/index.ts
packages/gitbook/public/~gitbook/static/
packages/icons/src/data/*.json
# Build files
dist/
+5
View File
@@ -0,0 +1,5 @@
{
"printWidth": 100,
"singleQuote": true,
"tabWidth": 4
}
-3
View File
@@ -1,3 +0,0 @@
{
"recommendations": ["biomejs.biome"]
}
+1 -11
View File
@@ -7,15 +7,5 @@
["style \\=([^;]*);", "\"([^\"]*)\""],
["style \\=([^;]*);", "\\`([^\\`]*)\\`"]
],
"tailwindCSS.classAttributes": ["class", "className", "style", ".*Style"],
"prettier.enable": false,
"editor.formatOnSave": true,
"editor.defaultFormatter": "biomejs.biome",
"editor.codeActionsOnSave": {
"source.organizeImports.biome": "explicit",
"source.fixAll.biome": "explicit"
},
"[typescript]": {
"editor.defaultFormatter": "biomejs.biome"
}
"tailwindCSS.classAttributes": ["class", "className", "style", ".*Style"]
}
-77
View File
@@ -1,77 +0,0 @@
# AGENTS Instructions
## Commands
```bash
bun install # Install dependencies
bun dev # Start dev server (all packages)
bun run build # Build all packages
bun run format # Format with Biome (run after every change)
bun run typecheck # Type-check all packages
bun run unit # Run unit tests
```
## Development
The dev server proxies published GitBook sites locally. After `bun dev`, access any published site at:
```
http://localhost:3000/url/<published-gitbook-url>
```
Examples:
- `http://localhost:3000/url/gitbook.com/docs`
- `http://localhost:3000/url/open-source.gitbook.io/midjourney`
## Architecture
```
packages/
gitbook/ # Main Next.js app
src/
app/ # Next.js App Router (sites/)
components/ # React components
lib/ # Server utilities, data fetching
intl/ # Internationalization (translations/)
openapi-parser/ # OpenAPI 3.0/3.1/Swagger parser
react-openapi/ # OpenAPI rendering components
react-contentkit/ # ContentKit component rendering
embed/ # Embeddable GitBook components
shared/ # Shared utilities
icons/ # Icon assets
fonts/ # Font assets
colors/ # Color tokens
expr/ # GitBook expression evaluator
cache-do/ # Cloudflare DO cache
cache-tags/ # Cache tag utilities
```
## Testing
```bash
bun run unit # Unit tests via bun test (not vitest)
bun run e2e # Playwright e2e tests (requires built app)
```
Run a specific test file:
```bash
cd packages/gitbook && bun test src/lib/cache.test.ts
```
## Changesets
After committing code changes, create a changeset for the affected package:
```md
---
"gitbook": patch
---
Provide a short description of the change.
```
Save as `.changeset/<name>.md`, then commit it separately with message: `changeset`
## Formatting
Uses Biome (not ESLint/Prettier). Always run `bun run format` before committing.
-1
View File
@@ -1 +0,0 @@
@AGENTS.md
+30 -11
View File
@@ -1,7 +1,7 @@
<h1 align="center">GitBook</h1>
<p align="center">
<a href="https://gitbook.com/docs/">Docs</a> - <a href="https://github.com/GitbookIO/community">Community</a> - <a href="https://developer.gitbook.com/">Developer Docs</a> - <a href="https://changelog.gitbook.com/">Changelog</a> - <a href="https://github.com/GitbookIO/gitbook/issues/new?assignees=&labels=bug&template=bug_report.md">Bug reports</a> - <a href="https://github.com/orgs/GitbookIO/discussions/categories/feature-requests">Feature requests</a>
<a href="https://docs.gitbook.com/">Docs</a> - <a href="https://github.com/GitbookIO/community">Community</a> - <a href="https://developer.gitbook.com/">Developer Docs</a> - <a href="https://changelog.gitbook.com/">Changelog</a> - <a href="https://github.com/GitbookIO/gitbook/issues/new?assignees=&labels=bug&template=bug_report.md">Bug reports</a>
</p>
<p align="center">
@@ -35,10 +35,8 @@ To run a local version of this project, please follow these simple steps.
### Prerequisites
- Node.js (Version: >= 22.3)
- Use nvm for easy Node management
- [Bun](https://bun.sh/) (Version: >=1.2.15)
- We use a text-based lockfile which isn't supported below 1.2.15
- Node.js (Version: >=18.x)
- Bun
### Set up
@@ -62,22 +60,43 @@ bun install
bun dev
```
6. Open a published GitBook space in your web browser, prefixing it with `http://localhost:3000/url`.
5. Open a published GitBook space in your web browser, prefixing it with `http://localhost:3000/`.
examples:
- http://localhost:3000/url/gitbook.com/docs
- http://localhost:3000/url/open-source.gitbook.io/midjourney
- http://localhost:3000/docs.gitbook.com
- http://localhost:3000/open-source.gitbook.io/midjourney
Any published GitBook site can be accessed through your local development instance, and any updates you make to the codebase will be reflected in your browser.
### Other development commands
- `bun format`: format the code
- `bun lint`: lint the code
### CI and testing
All pull-requests will be tested against both visual and performances testing to prevent regressions.
## Fonts and Icons
GitBook Open uses fontawesome. For self-hosting and local development, for licensing reasons, only the icons from the free version should be used.
GitBook Open uses fontawesome. During development, your local environment will use the free version. However, only the pro version will be accepted by CI. If you see the following error:
```
The GitBook icon is missing. It indicates that the dependencies were installed without the correct font-awesome package. These changes have probably been persisted in the Bun lockfile. Read the README for more information.
```
It means that you've changed the GBO dependencies and bundled in the free version. Only GitBook staff can help with this - if you're not on the GitBook team, please ping us in the PR and we'll help get things moving.
If you are GitBook staff, you'll need our NPM token in your local environment.
```
.env.local
NPM_TOKEN_READONLY=xxx
```
and then reinstall dependencies.
## Contributing
@@ -129,11 +148,11 @@ See `LICENSE` for more information.
</p>
```md
[![GitBook](https://img.shields.io/static/v1?message=Documented%20on%20GitBook&logo=gitbook&logoColor=ffffff&label=%20&labelColor=5c5c5c&color=3F89A1)](https://www.gitbook.com/preview?utm_source=gitbook_readme_badge&utm_medium=organic&utm_campaign=preview_documentation&utm_content=link)
[![GitBook](https://img.shields.io/static/v1?message=Documented%20on%20GitBook&logo=gitbook&logoColor=ffffff&label=%20&labelColor=5c5c5c&color=3F89A1)](https://gitbook.com/)
```
```html
<a href="https://www.gitbook.com/preview?utm_source=gitbook_readme_badge&utm_medium=organic&utm_campaign=preview_documentation&utm_content=link">
<a href="https://gitbook.com">
<img
src="https://img.shields.io/static/v1?message=Documented%20on%20GitBook&logo=gitbook&logoColor=ffffff&label=%20&labelColor=5c5c5c&color=3F89A1"
/>
Binary file not shown.

Before

Width:  |  Height:  |  Size: 491 KiB

After

Width:  |  Height:  |  Size: 622 KiB

-178
View File
@@ -1,178 +0,0 @@
{
"$schema": "https://biomejs.dev/schemas/1.9.4/schema.json",
"vcs": {
"enabled": false,
"clientKind": "git",
"useIgnoreFile": false
},
"files": {
"ignoreUnknown": false,
"ignore": [
"**/node_modules/**/*",
"**/dist/**/*",
"**/build/**/*",
"**/public/**/*",
"**/.next/**/*",
"**/.open-next/**/*",
"**/.turbo/**/*",
"**/.vercel/**/*",
"**/.cache/**/*",
"**/.wrangler/**/*",
"packages/embed/standalone/**/*",
"packages/openapi-parser/src/fixtures/**/*",
"packages/emoji-codepoints/index.ts",
"packages/icons/src/data/*.json",
"packages/gitbook/worker-configuration.d.ts",
"gitbook/tsconfig.json",
"**/*.css"
]
},
"formatter": {
"enabled": true,
"useEditorconfig": true,
"formatWithErrors": false,
"indentStyle": "space",
"indentWidth": 4,
"lineEnding": "lf",
"lineWidth": 100,
"attributePosition": "auto",
"bracketSpacing": true
},
"organizeImports": {
"enabled": true
},
"linter": {
"enabled": true,
"rules": {
"recommended": true,
"performance": {
"noDelete": "warn"
},
"security": {
"noDangerouslySetInnerHtml": "off"
},
"complexity": {
"noForEach": "off",
"noUselessFragments": "warn",
"noBannedTypes": "warn"
},
"correctness": {
"noUndeclaredVariables": "error",
"noUnusedVariables": "error",
"useArrayLiterals": "error",
"useHookAtTopLevel": "error",
"noUnusedImports": "error",
"noVoidElementsWithChildren": "warn",
"useJsxKeyInIterable": "warn",
"useExhaustiveDependencies": "warn",
"noUnknownFunction": "warn"
},
"style": {
"noNonNullAssertion": "warn",
"noParameterAssign": "off",
"useThrowOnlyError": "error"
},
"suspicious": {
"noConsole": {
"level": "warn",
"options": {
"allow": ["assert", "error", "warn"]
}
},
"noExplicitAny": "warn",
"noImplicitAnyLet": "warn",
"noConfusingVoidType": "warn",
"noControlCharactersInRegex": "warn",
"noPrototypeBuiltins": "warn",
"noAssignInExpressions": "warn",
"noArrayIndexKey": "warn"
},
"a11y": {
"useSemanticElements": "warn",
"useKeyWithClickEvents": "warn",
"noSvgWithoutTitle": "warn",
"useButtonType": "warn",
"useIframeTitle": "warn",
"useAltText": "warn",
"noPositiveTabindex": "warn",
"useFocusableInteractive": "warn",
"useAriaPropsForRole": "warn",
"useValidAnchor": "warn",
"noLabelWithoutControl": "warn",
"noNoninteractiveTabindex": "warn"
},
"nursery": {
"useSortedClasses": {
"level": "error",
"fix": "safe",
"options": {
"attributes": ["class", "className", "style"],
"functions": ["clsx", "tw"]
}
}
}
}
},
"javascript": {
"formatter": {
"jsxQuoteStyle": "double",
"quoteProperties": "asNeeded",
"trailingCommas": "es5",
"semicolons": "always",
"arrowParentheses": "always",
"bracketSameLine": false,
"quoteStyle": "single",
"attributePosition": "auto",
"bracketSpacing": true
}
},
"overrides": [
{
"include": [
"packages/gitbook/**/*",
"packages/react-openapi/**/*",
"packages/react-math/**/*",
"packages/react-contentkit/**/*",
"packages/icons/**/*"
],
"javascript": {
"globals": ["React"]
}
},
{
"include": ["packages/gitbook/**/*"],
"javascript": {
"globals": ["React", "GitBookIntegrationEvent"]
}
},
{
"include": ["*.css"],
"javascript": {
"globals": ["theme"]
}
},
{
"include": ["*.test.ts", "packages/gitbook/tests/**/*"],
"javascript": {
"globals": ["Bun"]
}
},
{
"include": [
"packages/cache-do/**/*",
"packages/gitbook/cf-env.d.ts",
"packages/gitbook/src/cloudflare-entrypoint.ts"
],
"javascript": {
"globals": [
"DurableObjectLocationHint",
"DurableObjectNamespace",
"DurableObjectStub",
"ContinentCode",
"Fetcher",
"ExportedHandler"
]
}
}
]
}
-4903
View File
File diff suppressed because it is too large Load Diff
Executable
BIN
View File
Binary file not shown.
+2
View File
@@ -0,0 +1,2 @@
[install.scopes]
"gitbook" = { token = "$NPM_TOKEN_READONLY", url = "https://registry.npmjs.org" }
+22
View File
@@ -0,0 +1,22 @@
# Caching
## Revalidating the cache
Invalidate cache can be done at two levels using tags:
- Data fetching cache
- Rendering cache
To invalidate and refetch the data cache, you can execute a POST request to `/~/gitbook/revalidate`:
```bash
curl --location --request POST 'https://gitbook/mycompany.com/~gitbook/revalidate' \
--header 'Content-Type: application/json' \
--data-raw '{"tags": ["space.id"]}'
```
To invalidate the rendering cache, the implementation mainly depends on the infrastructure serving the content, GitBook outputs a `Cache-Tag` header on every requests. The value of the header is a comma separated list of tags.
## Purging the cache
Purging the cache, without revalidating, is done by passing `"purge": true` in the request body.
+17 -46
View File
@@ -1,67 +1,38 @@
{
"name": "gitbook",
"version": "0.1.0",
"engines": {
"node": "^22.3.0"
},
"devDependencies": {
"@biomejs/biome": "^1.9.4",
"@changesets/cli": "^2.31.0",
"turbo": "^2.10.3",
"vercel": "50.37.3"
"@changesets/cli": "^2.27.7",
"prettier": "^3.0.3",
"turbo": "^2.1.2"
},
"packageManager": "bun@1.3.7",
"packageManager": "bun@1.1.18",
"overrides": {
"@codemirror/state": "6.4.1",
"@types/react": "catalog:",
"@types/react-dom": "catalog:",
"react": "catalog:",
"react-dom": "catalog:",
"esbuild": "0.27.3",
"axios": "1.8.4",
"@radix-ui/react-slot": "1.2.4"
"@codemirror/state": "6.4.1"
},
"private": true,
"scripts": {
"dev": "turbo run dev --concurrency 20",
"dev": "turbo run dev --ui=stream",
"build": "turbo run build",
"clean-deps": "rm -rf node_modules && rm -rf packages/*/node_modules",
"build:cloudflare": "turbo run build:cloudflare",
"lint": "turbo run lint",
"lint:fix": "turbo run lint -- --fix",
"typecheck": "turbo run typecheck",
"format": "biome check --write ./",
"format:check": "biome check --diagnostic-level=error ./",
"format": "prettier ./ --ignore-unknown --write",
"format:check": "prettier ./ --ignore-unknown --list-different",
"unit": "turbo run unit",
"e2e": "turbo run e2e",
"e2e-customers": "turbo run e2e-customers",
"changeset": "changeset",
"changeset-version": "changeset version && bun run format && bun update",
"publish-all-packages": "turbo run publish-to-npm --continue=dependencies-successful",
"release": "turbo run release && changeset publish",
"release:preview": "turbo run release:preview",
"download:env": "op read op://gitbook-x-dev/gitbook-open/.env.local >> .env.local",
"clean": "turbo run clean"
},
"workspaces": {
"packages": ["packages/*"],
"catalog": {
"@tsconfig/strictest": "^2.0.6",
"@tsconfig/node20": "^20.1.6",
"@gitbook/api": "0.186.0",
"@scalar/api-client-react": "^1.3.46",
"@types/react": "^19.0.0",
"@types/react-dom": "^19.0.0",
"assert-never": "^1.4.0",
"bidc": "^0.0.2",
"bun-types": "^1.1.20",
"classnames": "^2.5.1",
"react": "^19.0.1",
"react-dom": "^19.0.1",
"tsdown": "^0.15.6",
"typescript": "^5.5.3",
"usehooks-ts": "^3.1.1",
"vercel": "^50.26.1"
}
},
"workspaces": [
"packages/*"
],
"patchedDependencies": {
"decode-named-character-reference@1.0.2": "patches/decode-named-character-reference@1.0.2.patch",
"@vercel/next@4.4.2": "patches/@vercel%2Fnext@4.4.2.patch",
"next@16.2.6": "patches/next@16.2.6.patch"
"@vercel/next@4.3.15": "patches/@vercel%2Fnext@4.3.15.patch"
}
}
-52
View File
@@ -1,52 +0,0 @@
# @gitbook/browser-types
## 0.1.5
### Patch Changes
- 5f66860: Use isCookiesTrackingDisabled for cookie consent integrations
- 2e495cb: Add Global Privacy Control (GPC) support
## 0.1.4
### Patch Changes
- 7a11861: Add support for custom cookie banner registration
- Updated dependencies [6f1db32]
- @gitbook/icons@0.4.3
## 0.1.3
### Patch Changes
- 10995e0: Use NPM Trusted publishing for publishing the package.
- Updated dependencies [10995e0]
- @gitbook/icons@0.3.4
## 0.1.2
### Patch Changes
- 6142d6b: Mark as sideEffects, fix all package bundles
- Updated dependencies [6142d6b]
- @gitbook/icons@0.3.3
## 0.1.1
### Patch Changes
- 295f03d: Republish packages
- Updated dependencies [295f03d]
- @gitbook/icons@0.3.2
## 0.1.0
### Minor Changes
- cbc71a5: First version of the public package for typing script integrations.
### Patch Changes
- 854c448: Custom assistants followup
- Updated dependencies [25e2b40]
- @gitbook/icons@0.3.0
-3
View File
@@ -1,3 +0,0 @@
# `@gitbook/browser-types`
Typescript types for the global variables available in a GitBook website. These types can be used by integrations embedding scripts.
-36
View File
@@ -1,36 +0,0 @@
{
"name": "@gitbook/browser-types",
"description": "Typescript types for the global variables available in a GitBook website. These types can be used by integrations embedding scripts.",
"type": "module",
"exports": {
".": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
}
},
"sideEffects": false,
"version": "0.1.5",
"dependencies": {
"@gitbook/api": "catalog:",
"@gitbook/icons": "workspace:"
},
"devDependencies": {
"bun-types": "catalog:",
"tsdown": "catalog:",
"typescript": "catalog:"
},
"scripts": {
"build": "tsdown",
"typecheck": "tsc --noEmit",
"dev": "bun run build -- --watch ./src",
"publish-to-npm": "../../scripts/publish-if-new.sh"
},
"files": ["dist", "README.md", "CHANGELOG.md"],
"publishConfig": {
"access": "public",
"registry": "https://registry.npmjs.org/"
},
"repository": {
"url": "https://github.com/GitbookIO/gitbook"
}
}
-108
View File
@@ -1,108 +0,0 @@
import type { AIToolCallResult, AIToolDefinition } from '@gitbook/api';
import type { IconName } from '@gitbook/icons';
export type GitBookIntegrationEvent = 'load' | 'unload';
export type GitBookIntegrationEventCallback = (...args: any[]) => void;
export type GitBookIntegrationTool = AIToolDefinition & {
/**
* Confirmation action to be displayed to the user before executing the tool.
*/
confirmation?: {
icon?: IconName;
label: string;
};
/**
* Callback when the tool is executed.
* The input is provided by the AI assistant following the input schema of the tool.
*/
execute: (input: object) => Promise<Pick<AIToolCallResult, 'output' | 'summary'>>;
};
export type GitBookAssistant = {
/**
* Name of the assistant displayed in the UI.
*/
label: string;
/**
* Icon of the assistant displayed in the UI.
* Any FontAwesome icon name is supported.
* @example 'sparkle'
*/
icon: string;
/**
* Callback when the assistant is opened.
*/
open: (query?: string) => void;
/**
* Whether to display the triggers for this assistant in the UI.
* @default true
*/
ui?: boolean;
};
export type GitBookCookieBannerHandler = (options: {
onApprove: () => void;
onReject: () => void;
}) => void;
export type GitBookGlobal = {
/**
* Register an event listener.
*/
addEventListener: (
type: GitBookIntegrationEvent,
func: GitBookIntegrationEventCallback
) => void;
/**
* Remove an event listener.
*/
removeEventListener: (
type: GitBookIntegrationEvent,
func: GitBookIntegrationEventCallback
) => void;
/**
* Register a custom tool to be exposed to the AI assistant.
*/
registerTool: (tool: GitBookIntegrationTool) => void;
/**
* Register a custom assistant to be available on the site.
*/
registerAssistant: (assistant: GitBookAssistant) => () => void;
/**
* Register a custom cookie banner handler.
* When registered, the built-in cookie banner will not be displayed.
*/
registerCookieBanner: (handler: GitBookCookieBannerHandler) => void;
/**
* Indicates whether cookies tracking is disabled for the current user.
* Returns `true` when tracking is disabled, `false` when tracking is enabled,
* and `undefined` when the tracking preference is unknown or not yet determined.
*/
isCookiesTrackingDisabled: () => boolean | undefined;
/**
* Indicates whether global privacy control is enabled for the current user.
* Returns `true` when global privacy control is enabled, `false` when it is disabled.
*/
isGlobalPrivacyControlEnabled: () => boolean;
};
declare global {
interface Window {
/**
* Global `window.GitBook` object accessible by integrations.
*/
GitBook?: GitBookGlobal;
}
}
+3
View File
@@ -0,0 +1,3 @@
.wrangler
worker-configuration.d.ts
dist/
+18
View File
@@ -0,0 +1,18 @@
# @gitbook/cache-do
## 0.1.1
### Patch Changes
- b7a5106: Disable cloudflare observability in production
## 0.1.0
### Minor Changes
- 9b8d519: Experiment with optimizing billable duration in Cloudflare by using multiple RPC sessions instead of one
- 636b868: First version of a new cache backend powered by Cloudflare Durable Objects
### Patch Changes
- 56f5fa1: Enable Workers observability with a sampling of 0.1
+22
View File
@@ -0,0 +1,22 @@
# `@gitbook/cache-do`
Cache backend, powered by Cloudflare Durable Objects. The cache is optimized for GitBook use-cases.
### Performances
The cache backend is optimized for performances by being distributed and accessible close to the worker locations that are reading it.
### Geo-distribution
To achieve a good balance between **performances** and **consistency**, cache objects are distributed over 7 locations, representing continents.
It makes it possible to purge all 7 locations in one go and achieve fast consistency.
### Concepts
**Cache tag**: unique tag in the cache environment. A cache tag groups multiple keys that should be purged together in one operation.
Cache tags should not contain a large set of unique keys. Exceeding thousands could lead to performances or reliability issues.
**Cache key**: unique key in the cache environment. Each key should be assigned to a `tag`.
**Location**: cache is distributed over 7 unique locations, one for each continent.
+42
View File
@@ -0,0 +1,42 @@
{
"name": "@gitbook/cache-do",
"type": "module",
"private": true,
"exports": {
".": {
"types": "./dist/index.d.ts",
"development": "./src/index.ts",
"default": "./dist/index.js"
},
"./api": {
"types": "./dist/api.d.ts",
"development": "./src/api.ts",
"default": "./dist/api.js"
}
},
"version": "0.1.1",
"dependencies": {
"@msgpack/msgpack": "^3.0.0-beta2",
"lru_map": "^0.4.1"
},
"devDependencies": {
"typescript": "^5.5.3",
"wrangler": "^3.100.0"
},
"scripts": {
"generate": "wrangler types --experimental-include-runtime",
"build": "tsc",
"typecheck": "tsc --noEmit",
"dev": "tsc -w",
"release": "wrangler deploy",
"release:preview": "wrangler deploy && wrangler deploy --env preview"
},
"files": [
"dist",
"src",
"bin",
"data",
"README.md",
"CHANGELOG.md"
]
}
+299
View File
@@ -0,0 +1,299 @@
import { encode, decode } from '@msgpack/msgpack';
import { DurableObject } from 'cloudflare:workers';
import { LRUMap } from 'lru_map';
export interface CacheObjectDescriptor {
get: <Value = unknown>(key: string) => Promise<Value | undefined>;
set: <Value = unknown>(key: string, value: Value, expiresAt: number) => Promise<void>;
}
/**
* Value stored in a chunked binary msgpack format.
* Stored under the key `prop.${key}.${index}`.
*/
interface CacheObjectProp<Value = unknown> {
value: Value;
expiresAt: number;
}
/**
* Expiration clock stored under the key `exp.${expiresAt}.${key}`.
*/
interface CacheObjectExp {
/** Key of the property */
k: string;
/** Number of chunks */
c: number;
}
/**
* Durable Object class being deployed as a distributed cache.
*/
export class CacheObject extends DurableObject {
private lru = new LRUMap<string, { match: CacheObjectProp | undefined }>(500);
/**
* Open a descriptor to access the cache object.
* The goal is to minimize the amount of RPC sessions between the client and the cache object.
* One session is opened per request on the client side and used to perform multiple operations.
* https://developers.cloudflare.com/workers/runtime-apis/rpc/#return-functions-from-rpc-methods
*/
public open(): CacheObjectDescriptor {
return {
get: async <Value = unknown>(key: string) => {
return this.get<Value>(key);
},
set: async <Value = unknown>(key: string, value: Value, expiresAt: number) => {
await this.set(key, value, expiresAt);
},
};
}
/**
* Get the value of a property.
*/
public async get<Value = unknown>(key: string) {
return this.logOperation({ operation: 'get', key }, async (setLog) => {
// Try the memory state first.
const memoryEntry = this.lru.get(key);
if (memoryEntry) {
setLog({ memory: true });
setLog({ memoryMatch: !!memoryEntry.match });
if (!memoryEntry.match) {
return;
}
const isExpired = memoryEntry.match.expiresAt < Date.now();
setLog({ memoryExpired: isExpired });
if (!isExpired) {
return memoryEntry.match.value as Value;
}
}
return await this.getFromStorage<Value>(key);
});
}
/**
* Get the value of a property from the DO storage.
*/
public async getFromStorage<Value = unknown>(key: string) {
return this.logOperation({ operation: 'getFromStorage', key }, async (setLog) => {
const entries = await this.ctx.storage.list<Uint8Array>({
prefix: getStoragePropKey(key),
noCache: true,
});
if (entries.size) {
const entry = decodeChunks<CacheObjectProp<Value>>(entries);
setLog({ chunks: entries.size, chunksSize: entry?.size ?? 0 });
if (entry && entry.value.expiresAt > Date.now()) {
// Found
this.lru.set(key, { match: entry.value });
return entry.value.value;
}
}
// Not found
this.lru.set(key, { match: undefined });
});
}
/**
* Set a value in the cache object.
*/
public async set<Value = unknown>(key: string, value: Value, expiresAt: number) {
return this.logOperation({ operation: 'set', key }, async (setLog) => {
const prop: CacheObjectProp<Value> = {
value,
expiresAt,
};
this.lru.set(key, { match: prop });
await this.ctx.storage.transaction(async (tx) => {
const entries = encodeChunks(key, prop);
const chunks = Object.keys(entries).length;
setLog({ chunks });
const clockValue: CacheObjectExp = {
k: key,
c: chunks,
};
await tx.put(getGCClockKey(key, expiresAt), clockValue);
await tx.put(entries);
const currentAlarm = await tx.getAlarm();
if (!currentAlarm) {
// Set an alarm to garbage collect all entries that have expired in 12h.
await tx.setAlarm(Date.now() + 12 * 60 * 60 * 1000);
}
});
});
}
/**
* Purge all keys in the cache object.
*/
public async purge() {
return this.logOperation({ operation: 'purge' }, async (setLog) => {
let result = new Set<string>();
try {
// List all the keys in the cache object.
const entries = await this.ctx.storage.list<CacheObjectExp>({
prefix: 'exp.',
noCache: true,
});
setLog({ entries: entries.size });
entries.forEach((exp) => {
result.add(exp.k);
});
} catch (error) {
// If an error occurs, reset the cache object.
// This is a safety mechanism to prevent the cache object from being stuck in a bad state.
console.error('Error during purge, resetting the cache object', error);
}
await this.reset();
return Array.from(result);
});
}
/**
* Alarm to garbage collect all entries that have expired.
*/
async alarm() {
return this.logOperation({ operation: 'alarm' }, async (setLog) => {
try {
const entries = await this.ctx.storage.list<CacheObjectExp>({
prefix: 'exp.',
noCache: true,
});
setLog({ entries: entries.size });
const toDeleteSet = new Set<string>();
for (const [key, exp] of entries) {
const timestamp = parseInt(key.split('.')[1]);
if (timestamp < Date.now()) {
toDeleteSet.add(key);
for (let i = 0; i < exp.c; i++) {
toDeleteSet.add(getStoragePropChunkKey(exp.k, i));
}
}
}
// Delete the keys by batch of 128.
const toDelete = Array.from(toDeleteSet);
setLog({ toDelete: toDelete.length });
for (let i = 0; i < toDelete.length; i += 128) {
await this.ctx.storage.delete(toDelete.slice(i, i + 128));
}
// If there are still keys to delete, set an alarm to continue the deletion in 12h.
if (toDelete.length) {
await this.ctx.storage.setAlarm(Date.now() + 12 * 60 * 60 * 1000);
}
} catch (error) {
// If an error occurs, reset the cache object.
// This is a safety mechanism to prevent the cache object from being stuck in a bad state.
console.error('Error during alarm, reset the cache object', error);
await this.reset();
}
});
}
/**
* Reset the cache object.
*/
async reset() {
return this.logOperation({ operation: 'reset' }, async () => {
this.lru.clear();
await this.ctx.storage.deleteAll();
});
}
/**
* Time and log an operation.
*/
async logOperation<T>(
log: Record<string, unknown>,
fn: (update: (log: Record<string, unknown>) => void) => Promise<T>,
): Promise<T> {
const objectId = this.ctx.id.name ?? this.ctx.id.toString();
let update: Record<string, unknown> = {};
const start = performance.now();
try {
return await fn((arg) => {
Object.assign(update, arg);
});
} finally {
const duration = performance.now() - start;
console.log({ ...log, ...update, objectId, duration });
}
}
}
function getStoragePropKey(key: string): string {
return `prop.${key}.`;
}
function getStoragePropChunkKey(key: string, index: number): string {
return `${getStoragePropKey(key)}${index}`;
}
function getGCClockRootKey(timestamp: number): string {
return `exp.${timestamp}.`;
}
function getGCClockKey(key: string, expiresAt: number): string {
return `${getGCClockRootKey(expiresAt)}${key}`;
}
function encodeChunks<T>(key: string, value: T): Record<string, Uint8Array> {
const buf = encode(value);
const entries: Record<string, Uint8Array> = {};
const chunks = chunkUint8Array(buf, 128 * 1024);
for (let index = 0; index < chunks.length; index++) {
entries[getStoragePropChunkKey(key, index)] = chunks[index];
}
return entries;
}
function decodeChunks<T>(entries: Map<string, Uint8Array>): { value: T; size: number } | undefined {
const chunks = Array.from(entries.entries())
.map(([key, value]) => {
const index = parseInt(key.split('.').pop()!);
return [index, value] as const;
})
.sort(([a], [b]) => a - b)
.map(([, value]) => value);
if (chunks.length === 0) {
return;
}
const buf = mergeUint8Array(chunks);
return { value: decode(buf) as T, size: buf.length };
}
function chunkUint8Array(input: Uint8Array, chunkSize: number): Uint8Array[] {
const chunks: Uint8Array[] = [];
for (let i = 0; i < input.length; i += chunkSize) {
chunks.push(input.slice(i, i + chunkSize));
}
return chunks;
}
function mergeUint8Array(chunks: Uint8Array[]): Uint8Array {
const totalLength = chunks.reduce((sum, chunk) => sum + chunk.length, 0);
const result = new Uint8Array(totalLength);
let offset = 0;
for (const chunk of chunks) {
result.set(chunk, offset);
offset += chunk.length;
}
return result;
}
+97
View File
@@ -0,0 +1,97 @@
import type { CacheObject, CacheObjectDescriptor } from './CacheObject';
export type CacheLocationId = ContinentCode;
const allLocations: CacheLocationId[] = ['AF', 'AS', 'NA', 'SA', 'AN', 'EU', 'OC'];
/**
* Location hint for the CacheObject durable object.
*/
const doLocationHints: {
[key in CacheLocationId]: DurableObjectLocationHint;
} = {
AF: 'afr',
AS: 'apac',
NA: 'wnam',
SA: 'sam',
AN: 'oc',
EU: 'weur',
OC: 'oc',
};
/**
* Client to access a cache tag.
*/
export class CacheObjectStub {
private stub: DurableObjectStub<CacheObject>;
constructor(
/** Binding to the CacheObject durable object */
private doNamespace: DurableObjectNamespace<CacheObject>,
/** ID of the location to target */
private locationId: CacheLocationId,
/** Name of the tag */
private tag: string,
) {
const groupId = getCacheObjectIdName(this.locationId, this.tag);
this.stub = this.doNamespace.get(this.doNamespace.idFromName(groupId), {
// Initialize the object with a locaiton hint,
// as we might want to purge all locations before the object is created.
// https://developers.cloudflare.com/durable-objects/reference/data-location/
locationHint: doLocationHints[this.locationId],
});
}
/**
* Open a descriptor to the cache object.
* It can be used to perform multiple operations in a single RPC session.
* Ex:
* ```ts
* using desc = cache.open();
* await desc.set('key', 'value', Date.now() + 1000);
* await desc.get('key');
* ```
*/
async open() {
return await this.stub.open();
}
/**
* Get a value from the cache.
*/
async get<Value = unknown>(key: string) {
return (await this.stub.get(key)) as Value | undefined;
}
/**
* Set a value in the cache.
*/
async set<Value = unknown>(key: string, value: Value, expiresAt: number) {
return await this.stub.set(key, value, expiresAt);
}
/**
* Purge all keys in the cache tag.
*/
async purge() {
const keys = new Set<string>();
await Promise.all(
allLocations.map(async (locationId) => {
const groupId = getCacheObjectIdName(locationId, this.tag);
const cacheGroup = this.doNamespace.get(this.doNamespace.idFromName(groupId), {
// Initialize the object with a locaiton hint,
// as we might want to purge all locations before the object is created.
// https://developers.cloudflare.com/durable-objects/reference/data-location/
locationHint: doLocationHints[this.locationId],
});
const locationkeys = await cacheGroup.purge();
locationkeys.forEach((key) => keys.add(key));
}),
);
return keys;
}
}
function getCacheObjectIdName(locationId: CacheLocationId, tag: string): string {
return `${locationId}:${tag}`;
}
+1
View File
@@ -0,0 +1 @@
export * from './CacheObjectStub';
+9
View File
@@ -0,0 +1,9 @@
import { WorkerEntrypoint } from 'cloudflare:workers';
export * from './CacheObject';
export default class Worker extends WorkerEntrypoint {
fetch() {
return new Response('Hello, world!');
}
}
@@ -5,7 +5,6 @@
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"noUncheckedIndexedAccess": true,
"noEmit": false,
"declaration": true,
"outDir": "dist",
@@ -14,12 +13,9 @@
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "react-jsx",
"incremental": true,
"types": [
"bun-types" // add Bun global
]
"types": ["./.wrangler/types/runtime.d.ts"]
},
"include": ["src/**/*.ts", "src/**/*.tsx"],
"include": ["src/**/*.ts"],
"exclude": ["node_modules"]
}
+27
View File
@@ -0,0 +1,27 @@
main = "./src/index.ts"
name = "gitbook-open-cache"
compatibility_date = "2024-09-02"
durable_objects.bindings = [
{name = "CACHE", class_name = "CacheObject"}
]
migrations = [
{tag = "v1", new_classes = ["CacheObject"]}
]
[observability]
enabled = false
[env.preview]
name = "gitbook-open-cache-preview"
durable_objects.bindings = [
{name = "CACHE", class_name = "CacheObject"}
]
migrations = [
{tag = "v1", new_classes = ["CacheObject"]}
]
[env.preview.observability]
enabled = true
head_sampling_rate = 0.1
-1
View File
@@ -1 +0,0 @@
dist/
-43
View File
@@ -1,43 +0,0 @@
# @gitbook/cache-tags
## 0.3.4
### Patch Changes
- 10995e0: Use NPM Trusted publishing for publishing the package.
## 0.3.3
### Patch Changes
- 6142d6b: Mark as sideEffects, fix all package bundles
## 0.3.2
### Patch Changes
- 295f03d: Republish packages
## 0.3.1
### Patch Changes
- 77397ca: Fix version of @gitbook/api referenced in package.json
## 0.3.0
### Minor Changes
- 116575c: Improve typing of getComputedContentSourceCacheTags to match latest API specification
## 0.2.0
### Minor Changes
- f32bf1f: Export function `getCacheTagForURL` to easily get the cache tag for a URL.
## 0.1.0
### Minor Changes
- 05ffd0e: Initial version of the package
-3
View File
@@ -1,3 +0,0 @@
# `@gitbook/cache-tags`
Utility to generate cache tags for GitBook Open.
-35
View File
@@ -1,35 +0,0 @@
{
"name": "@gitbook/cache-tags",
"type": "module",
"exports": {
".": {
"types": "./dist/index.d.ts",
"default": "./dist/index.js"
}
},
"sideEffects": false,
"version": "0.3.4",
"dependencies": {
"@gitbook/api": "catalog:",
"assert-never": "catalog:"
},
"devDependencies": {
"bun-types": "catalog:",
"tsdown": "catalog:",
"typescript": "catalog:"
},
"scripts": {
"build": "tsdown",
"typecheck": "tsc --noEmit",
"dev": "bun run build -- --watch ./src",
"publish-to-npm": "../../scripts/publish-if-new.sh"
},
"files": ["dist", "README.md", "CHANGELOG.md"],
"publishConfig": {
"access": "public",
"registry": "https://registry.npmjs.org/"
},
"repository": {
"url": "https://github.com/GitbookIO/gitbook"
}
}
-201
View File
@@ -1,201 +0,0 @@
import type { ComputedContentSource } from '@gitbook/api';
import assertNever from 'assert-never';
/**
* Get a stringified cache tag for a given object.
*/
export function getCacheTag(
spec: /**
* All data related to a user
* @deprecated - in v2, no tag as this is an immutable data
*/
| {
tag: 'user';
user: string;
}
/**
* All data related to a space
*/
| {
tag: 'space';
space: string;
}
/**
* All data related to an integration.
*/
| {
tag: 'integration';
integration: string;
}
/**
* All data related to a change request
*/
| {
tag: 'change-request';
space: string;
changeRequest: string;
}
/**
* Immutable data related to a revision
* @deprecated - in v2, no tag as this is an immutable data
*/
| {
tag: 'revision';
space: string;
revision: string;
}
/**
* Immutable data related to a document
* @deprecated - in v2, no tag as this is an immutable data
*/
| {
tag: 'document';
space: string;
document: string;
}
/**
* Immutable data related to a computed document
* @deprecated - in v2, no tag as this is an immutable data
*/
| {
tag: 'computed-document';
space: string;
sourceType: string;
}
/**
* All data related to the URL of a content
*/
| {
tag: 'url';
hostname: string;
}
/**
* All data related to a site
*/
| {
tag: 'site';
site: string;
}
/**
* All data related to an OpenAPI spec
*/
| {
tag: 'openapi';
organization: string;
openAPISpec: string;
}
/**
* All data related to a translation
*/
| {
tag: 'translation';
organization: string;
translation: string;
}
): string {
switch (spec.tag) {
case 'user':
return `user:${spec.user}`;
case 'url':
return `url:${spec.hostname}`;
case 'space':
return `space:${spec.space}`;
case 'change-request':
return `space:${spec.space}:change-request:${spec.changeRequest}`;
case 'revision':
return `space:${spec.space}:revision:${spec.revision}`;
case 'document':
return `space:${spec.space}:document:${spec.document}`;
case 'computed-document':
return `space:${spec.space}:computed-document:${spec.sourceType}`;
case 'site':
return `site:${spec.site}`;
case 'integration':
return `integration:${spec.integration}`;
case 'openapi':
return `organization:${spec.organization}:openapi:${spec.openAPISpec}`;
case 'translation':
return `organization:${spec.organization}:translation:${spec.translation}`;
default:
assertNever(spec);
}
}
/**
* Get the tags for a computed content source.
*/
export function getComputedContentSourceCacheTags(
inContext: {
spaceId: string;
organizationId: string;
},
source: ComputedContentSource
) {
const tags: string[] = [];
if (!('dependencies' in source)) {
return tags;
}
// We add the dependencies as tags, to ensure that the computed content is invalidated
// when the dependencies are updated.
const dependencies = Object.values(source.dependencies ?? {});
if (dependencies.length > 0) {
dependencies.forEach((dependency) => {
switch (dependency.ref.kind) {
case 'space':
tags.push(
getCacheTag({
tag: 'space',
space: dependency.ref.space,
})
);
break;
case 'openapi':
tags.push(
getCacheTag({
tag: 'openapi',
organization: inContext.organizationId,
openAPISpec: dependency.ref.spec,
})
);
break;
case 'translation':
tags.push(
getCacheTag({
tag: 'translation',
organization: inContext.organizationId,
translation: dependency.ref.translation,
})
);
break;
default:
// Do not throw for unknown dependency types
// as it might mean we are lagging behind the API version
break;
}
});
} else {
// Push a dummy tag, as the v1 is only using the first tag
tags.push(
getCacheTag({
tag: 'computed-document',
space: inContext.spaceId,
sourceType: source.type,
})
);
}
// We invalidate the computed content when a new version of the integration is deployed.
if (source.type.startsWith('integration:')) {
const integration = source.type.split(':')[1]!;
tags.push(
getCacheTag({
tag: 'integration',
integration,
})
);
}
return tags;
}

Some files were not shown because too many files have changed in this diff Show More