Files
sencho/docs/features/git-sources.mdx
T
Anso f23b7e1bac feat: ordered multi-file Compose for Git sources (#1380)
* feat: ordered multi-file Compose for Git sources

Extend Git sources to deploy an ordered list of compose files merged with
docker compose -f base.yaml -f override.yaml ..., plus an optional project
directory.

- Pick and reorder compose files from the repository tree (drag to reorder on
  desktop, up/down arrows on phones); manual path entry is also supported.
- The ordered set drives every stack-scoped compose command (deploy, update,
  start/stop/restart/down, image scans, Compose Doctor) and the container
  lookup, so a service or image declared only in an override is handled too.
- Runtime keys off the materialized set, not the saved configuration: saving a
  source does not change deploy args until the pull is applied, and apply
  materializes from the pending snapshot rather than live config.
- The project directory is passed as --project-directory, with -p <stack>
  pinning the Compose project so container labels stay stable.
- The Mesh override is layered last; single-file sources are byte-identical to
  before, and existing rows keep working via the single-path fallback.

Docs cover the picker, ordering, project directory, and the new troubleshooting
and limitations (referenced files are not materialized; the dependency graph,
drift, and networking views read the primary file).

* fix: harden multi-file Git source (hash, unlink, collisions, node id)

- hashContent folds ordered file CONTENTS (not paths) so a clean multi-file
  stack is not flagged as locally edited: create/apply hash the fetched files
  (repo paths) while pull hashes the on-disk files (materialized paths), which
  previously disagreed and showed a false "local edits detected".
- Block unlinking a multi-file or project-directory Git source (409): the deploy
  spec lives on the source row, so removing it would silently revert deploys to
  root compose.yaml. Single-file sources still unlink.
- Reject materialized-path collisions in the selection validator: an additional
  file equal to or nested under compose.yaml, an ancestor/descendant overlap
  between selected files, and a project directory nested under a compose file
  (previously a 500 at materialization).
- DockerController.getContainersByStack uses the controller's node compose dir
  and passes its node id to the authored prefix, instead of the process default.

* fix: CI failures on multi-file Git source (test crash, aria query, path barrier)

- GitSourceFields no longer crashes when repoUrl/branch are falsy: the canBrowse
  trim() is optional-chained, so a reusable field component tolerates partial
  props. Fixes the apply-binding panel test, which feeds a minimal source object.
- GitSourcePanel tests query the footer Remove button by its exact name, so the
  picker's per-file "Remove <path>" buttons no longer collide with the broad
  /remove/i match (the test intent, footer Remove present/absent, is unchanged).
- validateCompose uses an inline resolve + startsWith barrier at the context-dir
  mkdir sink (CodeQL does not credit the wrapped isPathWithinBase helper),
  clearing the js/path-injection alert. The containment check is equivalent and
  contextDir is also validated upstream.

* test: update Git source E2E spec for the multi-file compose picker

The compose-file picker replaced the single #git-source-path input and added
per-file Remove buttons, so the E2E spec drove selectors that no longer exist:

- Drop the redundant compose.yaml fills (the picker defaults to compose.yaml).
- Select the footer Remove button by exact name so the picker's per-file
  "Remove <path>" buttons no longer make the locator ambiguous.
- Set a custom compose path through the picker (add via the manual input, press
  Enter, then remove the default compose.yaml).

* test: match the footer Remove button with an exact Playwright name

Playwright's getByRole name option is a substring match by default, so
{ name: 'Remove' } also matched the picker's "Remove <path>" buttons. Require an
exact match so only the footer Remove button is selected.
2026-06-17 13:24:55 -04:00

249 lines
20 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Git Sources
description: Link a stack to a Git repository and keep one or more compose files in sync via manual pulls or webhook triggers.
---
Git Sources turn any stack into a GitOps target. Point Sencho at a repository and branch, choose one or more compose files to merge in order, pull updates on demand or from CI, and review a diff before applying changes to disk. Optional sibling `.env` sync keeps configuration consistent too.
<Note>
Git Sources are available on every tier, including Community.
</Note>
## How it works
1. Open a stack and click the **Git Source** button in the editor toolbar.
2. Fill in the repository URL and branch, then choose the compose files. Use **Browse** to pick them from the repository tree, or type a path and press Enter. Add a token if the repo is private.
3. Click **Pull now** to fetch the latest commit. Sencho opens a side-by-side diff between the on-disk files and the incoming version.
4. Click **Apply** to write the incoming content to disk. Tick **Deploy after apply** in the same dialog to redeploy in one step.
Writes land in the stack's existing directory using the same storage Sencho uses for the in-browser editor.
## Anatomy of the panel
<Frame>
<img src="/images/git-sources/panel.png" alt="Git Source panel for a stack already linked to a repository, showing populated Repository URL, Branch, Compose file path, the Authentication toggle, the Apply behavior radio group, and a Last applied commit row at the bottom" />
</Frame>
The panel groups four regions:
- **Pending update banner.** Appears at the top when a webhook in **Review only** mode has fetched a new commit. Click **Review** to re-fetch the incoming commit and open the diff dialog.
- **Form fields.** Repository URL, branch, the ordered compose-file picker, an optional project directory, optional sibling `.env` sync, authentication toggle, and the apply behavior radio group.
- **Last applied stat strip.** Shows the short SHA of the last commit Sencho applied to disk, plus the timestamp of the most recent successful save or pull.
- **Footer actions.** **Remove** disconnects the source without touching the stack files; **Pull now** fetches the configured branch's HEAD; **Save** or **Update** persists form changes after a reachability check passes.
## Create a stack from a Git repository
Skip the "empty stack then link later" detour and point at a repo from the start. Click **Create Stack** in the sidebar, switch to the **From Git** tab, and fill in the same fields you would on the Git Source panel.
<Frame>
<img src="/images/git-sources/create-from-git-tab.png" alt="New stack dialog with the From Git tab selected, showing stack name, repository URL, branch, compose path, sibling .env toggle, authentication toggle, apply behavior radio group, a Deploy after create checkbox, and an HTTPS REPOS ONLY footer hint" />
</Frame>
Sencho fetches the compose files, validates the merged result with `docker compose config`, writes them to a fresh stack directory, and links the Git source in one step. The last-applied commit SHA is seeded from the fetch so the first manual pull produces a clean diff rather than a "local edits detected" warning.
Tick **Deploy after create** to run `docker compose up -d` immediately after the files land. If the deploy fails, the stack and Git source are kept on disk so you can fix the underlying issue (missing image, port conflict, host resources) and retry the deploy from the editor.
### Failure modes on create
| Situation | What happens |
|-----------|--------------|
| Stack name already exists | Sencho returns **409** with "Stack already exists" and makes no changes on disk or in the database. Pick a different name or remove the existing stack. |
| Repository unreachable or auth failed | Fetch fails before anything is created. The form stays open with an error toast describing the cause. |
| Fetched compose fails validation | The stack directory is not created and no Git source row is inserted. The error toast shows the `docker compose config` message. |
| Fetch + validate succeed but optional deploy fails | The stack and Git source are kept. The toast reads "Stack created, but deploy failed: ..." and you can retry the deploy from the editor. |
## Configure a source
| Field | Description |
|-------|-------------|
| **Repository URL** | `https://github.com/your-org/your-repo.git` (HTTPS only) |
| **Branch** | Branch to track (e.g. `main`) |
| **Compose files** | One or more paths within the repo, merged in the listed order (e.g. `deploy/base.yaml` then `deploy/prod.yaml`). The first file is the primary. Reorder by dragging (or the up/down arrows on a phone) and remove with the **×** button. |
| **Project directory** | Optional path within the repo passed to `docker compose --project-directory`, so relative build contexts, bind mounts, and `env_file` references resolve from that base. Leave blank to use the stack root. |
| **Also sync sibling `.env` file** | When enabled, also pulls the `.env` from the same directory as the primary compose file. The form shows the resolved path inline (e.g. `deploy/.env` for a primary at `deploy/compose.yaml`). |
| **Authentication** | **Public (no auth)** for public repos, **Personal Access Token** for private repos |
| **Apply behavior** | See the three modes below |
Saving runs a reachability check against the repository. If the URL is wrong, the token is invalid, the branch does not exist, or a file is missing, Sencho surfaces the error inline and nothing is persisted.
### Multiple compose files
Many projects split a stack into a base file plus environment overrides. List the files in the order you want them merged and Sencho deploys with `docker compose -f base.yaml -f prod.yaml ...`, applying each later file's values on top of the earlier ones, exactly as the Compose CLI does. The same ordered set is used for every action on the stack: deploy, update, restart, stop, and teardown.
On disk, the primary file lands as the stack's `compose.yaml` and each additional file keeps its repository-relative path inside the stack directory, so you can still browse and edit them in the file explorer. A single-file source behaves exactly as before.
### Apply behavior modes
| Mode | What happens when a webhook fires |
|------|-----------------------------------|
| **Review only** | Sencho fetches and validates the incoming commit and marks the stack as having a pending update. You review the diff and apply manually. |
| **Auto-write files** | Sencho writes the new compose and env to disk automatically but does not redeploy. Use this when another process handles rollout. |
| **Auto-deploy** | Sencho writes the files and immediately runs `docker compose up -d` so the stack picks up the new configuration. |
Auto-deploy implies Auto-write: you cannot deploy automatically without also writing the new files first.
You can always override on the spot: when you click **Apply** in the diff dialog, a **Deploy after apply** checkbox lets you deploy regardless of the configured mode.
## Pulling and reviewing changes
Click **Pull now** on the Git Source panel to fetch the latest commit on the configured branch.
<Frame>
<img src="/images/git-sources/diff-dialog.png" alt="GIT · PULL PREVIEW dialog for the demo-app stack, showing a Local edits detected on disk warning above a Monaco side-by-side diff between the on-disk compose.yaml and the incoming commit, with a Deploy after apply checkbox and an Apply button in the footer" />
</Frame>
The diff dialog shows:
- The `GIT · PULL PREVIEW` kicker and the short SHA of the incoming commit at the top.
- A side-by-side compare of the on-disk compose file and the incoming version.
- A `.env` tab when the source is configured to sync `.env`.
- An **Incoming compose failed validation** banner when the incoming compose fails `docker compose config`. The Apply button stays disabled until validation passes.
- A **Local edits detected on disk** banner when the on-disk content differs from the last applied commit. Applying in this state opens an **Overwrite local edits?** confirmation modal whose primary button is **Overwrite and apply**.
### Pending updates
When a webhook fires in **Review only** mode, the stack gets a pending GitBranch icon next to its row in the sidebar and a pulsing dot on the **Git Source** button in the editor. Clicking either re-fetches the commit and opens the diff dialog; the panel also shows a **Pending update** banner with a **Review** button.
If the same stack also has an image update available, the image-update dot in the sidebar takes priority over the Git source icon, so only the update dot renders. The pending Git source is still surfaced inside the editor on the **Git Source** button.
<Frame>
<img src="/images/git-sources/sidebar-badge.png" alt="Sidebar stack list with a small GitBranch icon next to the demo-app entry indicating a pending Git source update" />
</Frame>
Click **Dismiss** in the diff dialog to discard a pending update without applying.
## Trigger from CI with a webhook
Git sources integrate with Sencho's existing webhook system. Create a webhook targeting the stack with the **Git source sync** action.
<Frame>
<img src="/images/git-sources/webhook-action.png" alt="New webhook form with Name, Stack and Action fields. The Action select is open, showing Deploy, Restart, Stop, Start, Pull and Update, and Git source sync as options, with Git source sync highlighted at the bottom" />
</Frame>
The webhook's behavior on trigger depends on the source's apply mode:
- **Review only**: fetch, validate, diff, mark pending.
- **Auto-write files**: fetch, validate, write to disk.
- **Auto-deploy**: fetch, validate, write, deploy.
The Git source sync action is only selectable on webhooks whose target stack already has a Git source configured. Webhook triggers for a single source are debounced on a 10-second window so a runaway pipeline cannot overwhelm Sencho (or your repository host's rate limits); the dashboard records the skipped trigger in the webhook's execution history.
### GitHub Actions example
```yaml
- name: Sync compose via Sencho
run: |
BODY='{}'
SIGNATURE=$(echo -n "$BODY" | openssl dgst -sha256 -hmac "${{ secrets.SENCHO_WEBHOOK_SECRET }}" | cut -d' ' -f2)
curl -X POST "${{ secrets.SENCHO_URL }}/api/webhooks/${{ secrets.SENCHO_WEBHOOK_ID }}/trigger" \
-H "Content-Type: application/json" \
-H "X-Webhook-Signature: sha256=$SIGNATURE" \
-d "$BODY"
```
See the [Webhooks](/features/webhooks) page for the full signing protocol.
## Private repositories
For private repositories, use a Personal Access Token scoped to read access on the target repo:
- **GitHub**: a fine-grained PAT with **Contents: Read** permission on the repo, or a classic PAT with the `repo` scope.
- **GitLab**: a project or group access token with the `read_repository` scope.
- **Bitbucket**: an app password with **Repositories: Read**.
Paste the token into the **Token** field and save. Sencho stores it encrypted at rest and never returns it in API responses or UI. When editing the source later, the token field shows a masked placeholder; leave it blank to keep the stored value, or type a new token to replace it. Switching the auth type back to **Public (no auth)** clears the stored token.
The encryption boundary covers the pending update payload too: every pull caches the fetched compose and env content in the database so the diff dialog can reopen without a refetch, and that cached content is encrypted at rest in the same way as the token, since compose files routinely embed secrets via env interpolation.
## Local edits vs Git
Sencho tracks a hash of the compose and env contents at the moment of the last apply. When you pull, it compares that hash against the current on-disk content.
- Matching hash: applying overwrites content that Sencho itself last wrote.
- Differing hash: someone edited the files outside Git. The diff dialog shows the **Local edits detected on disk** banner, and Apply requires the **Overwrite local edits?** confirmation.
The in-browser editor and the Git Source panel both write to the same files, so you can always fall back to editing locally. The next pull will just flag the divergence rather than silently clobbering your edits.
Pulls, applies, and create-from-git operations on the same stack are serialized by a per-stack lock, so a webhook that fires in the middle of a manual apply waits for the apply to finish rather than racing it.
## Troubleshooting
<AccordionGroup>
<Accordion title="Repository not found or not accessible">
Verify the URL is reachable from the Sencho host and ends with `.git`. GitHub returns a "not found" response for both genuinely missing repos and private repos you cannot read, so Sencho tailors the hint based on what you provided:
- **No token configured**: the repo might be private. Switch **Authentication** to **Personal Access Token** and paste a token with read access.
- **Token configured**: double-check the URL is correct and the token has read access to this specific repo. GitHub fine-grained PATs need **Contents: Read** on the target repo; classic PATs need the `repo` scope.
</Accordion>
<Accordion title="Authentication failed">
You supplied a token and the Git host rejected it outright. The token is missing, expired, or lacks read access. Generate a new token and replace the value in the **Token** field. Sencho returns this as a 400 form error rather than a 401, so an upstream auth failure does not sign you out of the dashboard.
</Accordion>
<Accordion title="Branch not found">
The branch name is case-sensitive and must exist on the remote. Confirm the branch with `git ls-remote <url>` from a shell that has access.
</Accordion>
<Accordion title="File not found">
Each compose path is relative to the repository root and must point at the file, not its parent directory. Every file in the list must exist on the tracked branch; if any is missing, the save or pull fails and names the path. If a file was moved, update its path on the panel and save.
</Accordion>
<Accordion title="Compose validation failed">
Sencho runs `docker compose config` against the incoming content before letting you apply. The error banner shows the exact message. Common causes: unresolved `${VAR}` interpolation (commit a `.env` file next to the compose file and enable sibling `.env` sync), invalid `include:` paths, or schema issues introduced by a recent compose change. Validation has a 10-second budget; an unusually large compose with many services may need to be split.
</Accordion>
<Accordion title="Local edits detected">
The on-disk files diverge from the last applied Git commit. Either confirm **Overwrite and apply** to take the incoming content, or discard local work with a redeploy from the stack editor, or commit your local changes back to the repo so the diff becomes clean.
</Accordion>
<Accordion title="Webhook skipped (rate limited)">
Sencho debounces rapid-fire triggers on a 10-second window per source. Wait at least 10 seconds and retry, or consolidate multiple CI triggers into a single call at the end of your pipeline. Skipped triggers appear in the webhook's execution history.
</Accordion>
<Accordion title="Network timeout">
The clone did not finish in time. Fetches run with a 30-second timeout to keep a slow or unreachable host from hanging the stack panel. Check that the Sencho host can reach the repository host (proxies, firewalls, DNS) and try again. If the repository is genuinely large, pin a smaller compose subpath or mirror it somewhere closer to the Sencho host.
</Accordion>
<Accordion title="Repository too large">
A clone is capped on how much it downloads (100 MB by default), and individual compose/env files are capped on read. A compose repository is normally tiny, so hitting either usually means the tracked branch carries large binaries. Point the source at a repository or branch that holds just your compose and `.env` files. If a large repository is unavoidable, an operator can raise the download ceiling with the `GITSOURCE_MAX_CLONE_BYTES` environment variable.
</Accordion>
<Accordion title="Pending commit has changed since this pull was fetched">
You opened a diff dialog, then a webhook fired and replaced the pending commit before you clicked **Apply**. Close the dialog and reopen the panel to load the latest pending commit; the **Review** button will fetch the newer one.
</Accordion>
<Accordion title="Applied but deploy failed">
The incoming compose file was written to disk successfully, but the subsequent `docker compose up -d` did not complete. The toast message includes the underlying reason (for example, an image pull failure or a port conflict). The stack is already on the new content, so you can retry the deploy directly from the editor's **Deploy** button without re-pulling. Fix the root cause first (image availability, host resources, network config) and redeploy.
</Accordion>
<Accordion title="Stack contents look wrong or appear empty">
If the compose file in your repository is tracked via Git LFS, Sencho will refuse the link and surface an LFS error rather than write a pointer stub as real content. Commit the plain compose file (and any synced `.env`) without LFS, or replace the LFS pointer in-place, then retry.
</Accordion>
<Accordion title="A build context or volume points at an empty folder">
Repositories that use Git submodules do not have their submodule contents cloned during a Git Source fetch. Sencho surfaces a warning on create when `.gitmodules` is present. If the compose file references paths inside a submodule (build contexts, volume mounts, include directives), inline the referenced files into the main repository or flatten the submodule so the paths resolve at deploy time.
</Accordion>
<Accordion title="An additional file named compose.yaml is rejected">
The first file in the list is always written to the stack's root `compose.yaml`. To avoid clobbering it, an additional file whose repository path is also `compose.yaml` is rejected. Rename it in the repository, or move it to the top of the list to make it the primary.
</Accordion>
<Accordion title="A relative build context or extends target is missing with multiple files">
A Git Source materializes only the compose and `.env` files you select, not the rest of the repository. Build contexts, bind-mount sources, and cross-file `extends:` targets referenced by relative path are not pulled, so they can be missing at deploy time. Set a **Project directory** to fix the base for relative paths, and for build-based stacks keep the referenced files in the repository alongside the compose files or build the image out of band.
</Accordion>
<Accordion title="Only HTTPS is supported">
Git Sources fetch over HTTPS only. SSH clone URLs (`git@host:org/repo.git`) and custom protocols are rejected with a client-side validation error. Paste the `https://...` URL and use a Personal Access Token for authentication on private repositories.
</Accordion>
</AccordionGroup>
## Known limitations
- **HTTPS only.** SSH URLs and SSH keys are not supported. Use a Personal Access Token for private repos.
- **No Git LFS.** Compose and env files stored via LFS are rejected. Commit plain files instead.
- **No submodules.** Submodule contents are not fetched; paths inside a submodule directory will be missing at deploy time. A warning is shown on create when `.gitmodules` is present.
- **Branch-tracking only.** Sources follow the head of a branch. Specific commit SHAs and tags are not pinnable.
- **Clone size cap.** A clone is bounded on how much it downloads (and each compose/env file is capped on read), so very large repositories are rejected. Operators can adjust the download ceiling with `GITSOURCE_MAX_CLONE_BYTES`.
- **Referenced files are not materialized.** Only the selected compose and `.env` files are written to disk. Relative build contexts, bind-mount sources, and `extends:` targets are not pulled, so build-based stacks that depend on other repository files need those files committed alongside the compose files (or the image built out of band).
- **Some read-only views read the primary file.** The dependency graph, drift snapshot, and networking inspector summarize the primary compose file, so a service declared only in an override may not appear in those views. Deploy, update, image-update checks, and mesh attachment use the full merged set.