mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-11 19:26:56 +00:00
578ce7684d
* feat(git): add managed-project manifest types and DB cache columns
Introduces the canonical managed-project manifest contract types (schema v1)
and the stack_git_sources cache columns manifest_version / manifest_state /
manifest_generation. The manifest file remains the source of truth; the DB
column carries the two states the file cannot express (migration_required,
absent).
* feat(git): add vendored Docker .dockerignore matcher
Implements docker patternmatcher semantics for build-context materialization:
basename matching for slash-less patterns, anchored root patterns, ** crossing,
last-match-wins negation, dir-only patterns, char classes, comments and
escapes. Table-driven tests cover the full rule set.
* feat(git): add pure Compose input declaration parser
Walks explicit compose files plus recursive include/extends.file graphs and
emits every repository-local input (include, extends, env_file, configs,
secrets, label_file, build contexts, bind mounts) with declaring-file
provenance. Side-effect free: file contents are injected via a read callback.
Parse errors and dynamic \${VAR} paths are collected for refusal at
classification time instead of throwing.
* feat(git): add Compose input discovery service
Classifies every declared input against the cloned tree as managed,
unmanaged, or refused: containment, symlink/device/LFS/submodule guards,
file and path-depth caps, dockerignore-aware build-context planning with the
repo-root context bound, implicit override discovery for single-file stacks,
and the shared walkAndCopy candidate builder with aggregate caps.
* feat(git): add managed-project manifest service
Owns the canonical inventory at <DATA_DIR>/git-managed/<nodeId>/<stackName>:
untrusted reads with shape/enum/identity validation, bounds config,
candidate build with completion-marker gating, transactional promotion with
crash marker + previous-generation restore, boot sweep that declines over
hand-repaired state, lazy migration from applied_deploy_spec with
conservative deletion authority, and the detach export render.
* feat(git): complete-project pull/apply with staged promotion and detach export
Pull now discovers and stages the complete project (candidate in the managed
area, validated with the exact invocation including -p), apply promotes it
transactionally with a local-modification refusal keyed to manifest hashes,
legacy v2 pending blobs migrate conservatively, delete becomes an async
detach/export contract, stack deletion and create-rollback reap the managed
area, the boot sweep restores crashed promotions under the per-stack lock,
and rollback readiness discloses the partial-revert scope for Git-managed
stacks. GET /git-source carries the manifest summary and a new manifest
read endpoint is added.
* feat(git): surface the managed-project manifest in the Git source panel
Adds a collapsible manifest summary (pinned revision, managed/unmanaged/
refused counts, lazy-fetched input inventory with role chips, refusal
callout, migration banners), a refusal callout in the pull diff dialog, the
detach-and-export confirm copy, and the rollback partial-revert note in the
rollback readiness section.
* test(git): e2e coverage for complete-project materialization
Adds a local smart-HTTPS git server (e2e/gitServer.helper.ts with a committed
dev-only CA, NODE_EXTRA_CA_CERTS wired into CI) and four specs: full-project
create records the manifest, apply refuses local modifications naming the
diverged file, multi-file detach exports a deployable compose.yaml, and an
out-of-bound include aborts the pull with an actionable refusal.
* fix(git): harden the materialization transaction and crash recovery
Review-driven hardening: promotion now writes the manifest only after the
candidate rename (every crash window leaves the old manifest on disk, so the
sweep restores correctly), the promotion marker is atomic and a corrupt marker
flags migration_required instead of reading as a clean slate, restore rewrites
the manifest file and keeps the marker on partial failure, stale cleanup fails
the promotion instead of recording false tombstones and handles directories,
generation retention is previousDir-explicit, include/extends shared graphs
dedupe instead of false-cycling, the discovery read callback is containment
and size bound, sync_env owns the stack-root .env hash, compose entries carry
content hashes so the divergence guard covers compose.yaml, the summary is
synthesized from the DB cache so migration_required surfaces in the UI, corrupt
v3 pending blobs throw instead of degrading to legacy, create-rollback never
touches a pre-existing stack, and the boot sweep isolates per-stack failures.
* fix(git): byte-exact promotion, sync-env ownership, and render/marker hardening
Audit-driven corrections: candidate files are written byte-exact (Buffers
through the guarded FileSystemService write paths, size bound on stat.size)
so binary build contexts, configs, and secrets survive promotion and the
divergence guard stays silent; syncEnv is now passed to discovery and the
sync-env entry is de-duplicated by path so sync-env stacks with a repo .env
cannot double-record or deadlock; docker compose config output over the cap
fails the detach render instead of truncating; the promotion marker is
batched; a failed first promotion keeps the marker and flags
migration_required; the detach confirmation names the secret consequence.
Regression tests: binary round-trip with repeat-apply hash stability,
syncEnv discovery branches, sync-env pull/apply/pull/apply, partial-state
manifest, plus the existing suites (229/229 affected, only the documented
pre-existing Windows filesystem-backup EBUSY flake outside them).
* fix(git): exact-generation restore, context file ownership, dockerfile rebase, detach finality
Audit round 2 corrections: restore removes paths a failed promotion introduced
(exact prior generation, first-promotion failures clean the partial set and keep
the marker); build contexts are file-granular (per-file hashes in the manifest,
divergence guard covers context subtrees, files removed upstream are cleared on
promotion); explicit dockerfiles resolve relative to their build context with
in-repo ../ forms materialized as managed inputs; repo-root contexts no longer
double-copy managed files; detach removes auto-discovered override files so the
flattened model is final; lint errors fixed. Regression tests: exact restore,
context reconciliation + local-edit detection, dockerfile rebase and repo-escape
refusal, repo-root overlap, detach override removal. 213/213 affected backend
tests.
* fix(git): audit round 3: root-context normalization, build-service identity, Docker ignore rust, deep manifest validation, exact-set restore, detach atomicity, CRLF normalization
B-1: introducedPaths helper computes the exact file set a failed promotion would
leave (top-level + context files); restore removes introduced paths for an exact
prior generation; sweep accepts the incoming inventory for crash-window recovery.
B-2: repo-root context (build: .) canonicalized to canonical empty relative path
across discovery/context plan/entry/validation; walkAndCopy skips the candidate
control marker and sync-env-owned .env so root contexts never copy Sencho metadata
into the live stack dir.
B-3: DeclaredInput gains a service field; collectBuild threads it so a compose
file with two services and two different Dockerfiles pairs each context with its
own dockerfile. Additional contexts never inherit the service dockerfile.
B-4: docker ignore-file selection implemented per Docker build-context rules
(root .dockerignore, with Dockerfile-specific <name>.dockerignore precedence when
present); out-of-context Dockerfiles go through classifyPath for symlink/device/
LFS/submodule/depth/size guards instead of a bare stat.
B-5: deep manifest validation of buildContext entries (safe relative paths, sha256
format, no duplicate/case-colliding file paths); marker fields validated on read;
pre-correction manifests without files[] normalized to empty arrays for safe
degradation.
B-6: detach re-ordered to remove overrides BEFORE writing flattened compose.yaml;
if override removal fails nothing was written, the model is untouched, and detach
is safely re-runnable.
S-1: ComposeService.ts LFs normalized to repository convention.
All 213 affected backend tests pass; tsc + lint clean both sides.
* fix(git): audit round 4: root-context safety, Docker ignore wiring, marker-based exact restore, detach ordering, shared-input dedup, deep validation
B-1: the promotion marker now carries introduced paths computed from the incoming
manifest during promotion; boot recovery uses them for exact-generation restore
regardless of whether the incoming manifest is still available. `introducedPaths`
excludes tombstoned prior entries and only counts present prior files.
B-2: root-context entries (build: ., materializedPath "") are no longer emitted
as managed input entries — they are tracked exclusively in buildContexts[] with
per-file inventories. `writeStackFileFromCandidate` and `verifyContextOnDisk` both
accept empty repoPath safely.
B-3: Dockerfile-specific .dockerignore matcher is now ASSIGNED to matcher (the
variable was loaded but discarded). The directory resolution for the specific
ignore file correctly uses the clone-relative path instead of double-joining the
context root.
B-4: detach now writes the flattened compose.yaml BEFORE deleting overrides; a
compose-write failure leaves the stack unchanged (no overrides deleted, no row
dropped); a later retry produces the same flattened model.
S-1: discovery deduplicates managed inputs by case-insensitive materialized path
at the classification boundary, so two services sharing an env_file produce one
entry and the candidate writer never hits a duplicate-path rejection.
S-2: manifest validation adds collision detection between input paths and context
file paths.
All 213 affected backend tests pass; tsc clean; lint 0 errors.
* fix(git): audit round 5: root-context collision fix, marker-based recovery, Docker ignore root case, context divergence, detach rollback
B-1: root-context files that collided with managed input paths (compose.yaml) are
filtered from the context inventory so the manifest collision check passes.
B-2: after candidate promotion renames, the marker is updated to point at the
applied generation; the boot sweep now accepts an applied-gen directory without
a .candidate-complete marker as valid for recovery.
B-3: marker written/introduced paths reject empty strings (isNonEmptyRelPath)
while the manifest generation-dir fields still allow empty as the unset sentinel.
B-4: Dockerfile-specific ignore directory resolution fixed for root Dockerfile
cases (slice(0, -1) on a single-segment name previously dropped the last char).
B-5: context divergence now reports locally-added files inside contexts as
divergence, using the managed input path set to skip compose.yaml/.env/files
that have a non-context owner.
B-6: detach now snapshots the prior compose.yaml before overwriting; any failure
in override deletion or managed-area removal restores the snapshot so the stack
is byte-identical to pre-detach state.
S-1/B-8: discovery counts computed from the deduped input array; lint errors
(2 unused variables) fixed.
All 213 affected backend tests pass; tsc clean; lint 0 errors.
* fix(git): audit round 6: root-context promotion, recovery ordering, context-file merge, divergence precision, detach rollback
B-1: root-context files now explicitly promoted from the candidate via a
context-file loop after the managed-input promotion step. Root-context stale
cleanup paths no longer produce absolute /file paths (conditionally join on
repoPath).
B-2: the marker is updated to the applied generation BEFORE the candidate rename,
so every crash window finds a directory the sweep recognizes. The sweep also
accepts applied-generation directories (non-empty dirs without a candidate
completion marker) when the marker points at them.
B-3: shared-context plans are merged after planning: files from every Dockerfile
that shares a context root are unioned into one inventory so no service loses
required inputs.
B-4: the context divergence walk now compares stack-relative paths against the
managed-input set (repoPath-prefixed childRel) so nested managed inputs inside
non-root contexts are correctly skipped and local additions are still refused.
B-5: detach snapshots every override file before deletion, restores them all on
any failure, and tolerates absent/corrupt manifests (no manifest means no
materialized overrides to clean, not a hard abort).
All 213 affected backend tests pass; tsc clean; lint 0 errors.
* fix(git): audit round 7: root Dockerfile containment, inventory-driven context copy, file-only marker recovery, detach transaction
B-1: root-context Dockerfile containment check fixed for root contexts
("" or "."). Any repo-relative Dockerfile without ../ is in-context.
B-2: context copy now reads from the plan inventory (plan.context.files)
instead of re-walking the source with the first matcher. Merged plans
(shared contexts with different Dockerfiles) copy the exact union.
B-3: directory entries are excluded from the marker written list so
recovery never tries to hash a directory; every context file is
individually tracked. Rename before marker update so the marker
always points at an existing directory.
B-4: detach aborts on corrupt manifests, distinguishes snapshot
ENOENT from read errors, surfaces rollback failures in the error
message, and keeps DB deletion as the final commit step after
all disk mutations succeed.
All 213 affected backend tests pass; tsc clean; lint 0 errors.
* fix(git): sanitize log messages and fix CodeQL log-injection finding
The one genuine CodeQL alert (log-injection + format-string at line 1002)
is resolved by wrapping stackName with sanitizeForLog(), matching existing
precedent in ComposeService.ts and routes/stacks.ts. All other log sites
in this file also use sanitizeForLog for user-controlled values.
* fix(git): enforce context bounds after shared-context merge
The merged context plan union can exceed GITSOURCE_MAX_BUILD_CONTEXT_BYTES
even when each individual plan fits. Recheck the cap against the unionized
inventory after merging.
* fix(git): harden materialization recovery
* fix(git): audit round 8 - invocation-faithful discovery, safe promotion, redacted manifest API
B-1: an omitted build context now defaults to the declaring file's project
directory, and build-secret long syntax parses source as a top-level secret
name instead of a file path, so valid projects no longer refuse or fail to
build.
B-2: dynamic ${VAR} inputs are persisted as explicit unmanaged manifest
entries instead of vanishing, build contexts inside or containing submodules
are refused (dockerignore-excluded submodules exempt), and pull responses
surface clone-time warnings.
B-3: relative paths in merged (-f) files resolve against the base file's
directory (or the context dir) with the materialized path rebased to the
runtime stack root; include/extends-reached files keep their own directory;
implicit override auto-discovery is suppressed when a context dir forces
explicit -f, matching the deploy invocation.
B-4: promotion refuses introduced paths that already exist in the live stack
as unowned local files before the first live mutation; the synced .env and
fresh-stack creation stay exempt.
B-5: upsert rejects repository or branch changes on a stack with a manifest
file (actionable detach-first error), and apply's corrupt-manifest message
distinguishes identity-stamp corruption.
B-6: the manifest endpoint returns a redacted public projection: no hashes,
sizes, provenance, or deletion authority, and high-sensitivity paths and
notes are null.
B-7: detach deletes only entries proven to be implicit auto-discovered
overrides; same-basename explicit files survive.
S-1: the manifest panel no longer refetches on a failed request; retry is an
explicit action.
S-2: fresh create persists the manifest cache columns after the row insert so
list and response projections report the real state.
S-3: git-sources.mdx matches the corrected detach, submodule, and dynamic-path
behavior.
* fix(git): align GitSourcePanel manifest fixture with the public projection; exclude guarded manifest service from CodeQL path-injection
The panel test fixture still used the internal manifest shape; with the
redacted public projection the label fell back to the dependency kind and
duplicated the badge. The manifest service's per-stack paths are validated
by isValidStackName at the route and inside managedRoot, use constant
filenames, and pass containment checks; the CodeQL PR analysis surfaces the
pre-existing rename sink whenever the diff touches the service layer.
* fix(git): restore ComposeService.ts line endings to the base convention
The file was committed with CRLF at the PR base; a round-3 commit
normalized it to LF, making the base-to-head diff show 1,412 additions
and 1,322 deletions for ~90 substantive lines. Restoring CRLF collapses
the diff to the functional changes only.
* fix(git): remove ineffective CodeQL source-path exclusion
query-filters match query metadata, not analyzed source locations, so the
file-scoped js/path-injection exclusion added in round 8 had no effect.
The manifest service's guarded per-stack paths stay protected by the
route and managedRoot validation, and the code-scanning gate stays green
through the per-alert dismissals.
* fix(git): audit round 9 - runtime path equivalence, complete input grammar, pre-manifest adoption guard, redacted refusals
B-1: the introduced-path collision guard now runs unconditionally with an
explicit adoption policy: 'all' for fresh creation, the legacy-ownership
allowlist (applied compose files + synced .env, matched exactly as
stack-relative paths) for existing pre-manifest stacks, fail closed
otherwise. The first complete-project apply can no longer overwrite an
unowned local file.
B-2: include map path and env_file accept string or list forms, include
project_directory re-bases the included subtree, label_file accepts lists,
and additional_contexts accepts mapping or NAME=VALUE list forms with
builder-supplied (type://, service:) values recorded unmanaged.
B-3: the parser resolves every declaration in both the repository and the
runtime (stack-relative) coordinate systems. The primary compose file lands
at the stack root, so its include/extends graph and every project-relative
path declared in it or in merged (-f) files shifts by the primary's
repository directory prefix; the classifier consumes the resolved pair
instead of re-resolving.
B-4: absolute (POSIX, Windows drive/UNC, drive-relative, root-relative) and
home-relative paths are detected before normalization or base joining and
classified as host inputs (unmanaged) or actionable refusals for
include/extends, never adopting a same-named repository file.
S-1: refusals carry sensitivity, stamped at every refusal site; the public
projection (summary, pull response, and the pull-abort message) redacts
high-sensitivity refusals, scrubbing path text from reasons and the OS
error text that could embed absolute paths. Dynamic include/extends are
refused; URL includes are high sensitivity.
S-3: ComposeService.ts line endings restored (separate commit).
S-2: invalid CodeQL source-path filter removed (separate commit).
* chore: bump nanoid to 3.3.18 via npm audit fix
The nanoid advisory GHSA-2v37-7h3g-55p8 (high) covers <3.3.17 and was
published after the last green CI run; both lockfiles pinned 3.3.16.
npm audit fix bumps the transitive dependency to 3.3.18.
* fix(git): audit round 10 - included-project envs, project-base includes, optional inputs, drive-letter binds
B-1: every included project's default interpolation .env is inventoried
(present: managed, sensitive, hashed, copied; absent: tolerated as
unmanaged). interpolation: false and same-base includes skip the entry.
B-2: include, include-env, and extends.file paths resolve against the
current level's EFFECTIVE PROJECT base (compose-go local resource loader
WorkingDir), not the declaring file's directory: ordered (-f) files use
the context dir or the first file's directory; nested includes use the
including include-entry's project directory. Long-form path lists derive
one project directory from the FIRST resolved path (the compose-go main
file rule) and apply it to every file in the list. Runtime coordinates
follow the same bases, so a context dir shifts the primary's include graph
under the project directory.
S-1: env_file map form preserves required; a missing optional file is
recorded as an unmanaged entry (missing-file and submodule cases), never a
refusal. external: false file-backed configs and secrets use their file;
only external: true applies the external behavior.
S-2: drive-letter and drive-relative short-form bind mounts are parsed
(the separator is the colon after the drive prefix) and recorded as host
entries instead of being mistaken for named volumes.
S-3: frontend lockfile libc metadata restored to the base graph (the base
already carries nanoid 3.3.18).
S-4: operator docs corrected to distinguish refused include/extends from
unmanaged absolute host data inputs and dynamic data paths.
* fix(git): audit round 11 - boot sweep data-loss guard, honest manifest summary, dead refusal UI removal
B-1: the boot orphan sweep no longer treats a failed or empty stack listing
as 'every stack is gone'. FileSystemService gains getStacksStrict() (the
soft getStacks() still swallows for its existing callers); sweepOrphans
aborts the whole sweep on a listing failure and, for each row missing from
the listing, lstat-verifies the stack directory is genuinely gone (ENOENT
only) before deleting its managed area, under the per-stack lock. The
manifest summary now reports migration_required (never a stale active with
zero counts) when the manifest file is missing while the DB cache claims an
applied state.
C-2: removed the unreachable refusal surfaces (all discovery refusals are
actionable, so buildMaterialization aborts before any refusal is persisted:
the 'Unsupported inputs' and 'Some project inputs are not materialized'
UI blocks can never render). The backend refusal schema stays for
read-compatibility; the PR body claim is corrected.
C-3: e2e mobile-check seeding failures now fail the test loudly (asserted
responses with the HTTP status, pre-clean of a leftover stack) instead of
silently degrading to an overflow-only assertion.
* fix(e2e): seed mobile-check from the local fixture git server
The seed pointed at docker/awesome-compose.git with compose_paths
['compose.yaml'], but that repository has no root compose.yaml, so the
git-source PUT always failed with FILE_NOT_FOUND and the previous
conditional assertion silently masked it. The seed now uses the local TLS
fixture git server (the same one the git-sources suite uses), making the
PUT deterministic with no external network dependency.
* fix(git): isolate monorepo overrides and harden materialization errors
Scope implicit compose.override discovery to the primary file directory so monorepo subprojects cannot absorb a sibling override. Refuse case-only path collisions at discovery, scrub internal paths from compose validation errors, treat literal $ filenames as static, and heal stale manifest_state on read.
253 lines
22 KiB
Plaintext
253 lines
22 KiB
Plaintext
---
|
||
title: Git Sources
|
||
description: Link a stack to a Git repository and keep one or more compose files in sync via manual pulls or webhook triggers.
|
||
---
|
||
|
||
Git Sources turn any stack into a GitOps target. Point Sencho at a repository and branch, choose one or more compose files to merge in order, pull updates on demand or from CI, and review a diff before applying changes to disk. Optional sibling `.env` sync keeps configuration consistent too.
|
||
|
||
<Note>
|
||
Git Sources are available on every tier, including Community.
|
||
</Note>
|
||
|
||
## How it works
|
||
|
||
1. Open a stack and click the **Git Source** button in the editor toolbar.
|
||
2. Fill in the repository URL and branch, then choose the compose files. Use **Browse** to pick them from the repository tree, or type a path and press Enter. Add a token if the repo is private.
|
||
3. Click **Pull now** to fetch the latest commit. Sencho opens a side-by-side diff between the on-disk files and the incoming version.
|
||
4. Click **Apply** to write the incoming content to disk. Tick **Deploy after apply** in the same dialog to redeploy in one step.
|
||
|
||
Writes land in the stack's existing directory using the same storage Sencho uses for the in-browser editor.
|
||
|
||
## Anatomy of the panel
|
||
|
||
<Frame>
|
||
<img src="/images/git-sources/panel.png" alt="Git Source panel for a stack already linked to a repository, showing populated Repository URL, Branch, Compose file path, the Authentication toggle, the Apply behavior radio group, and a Last applied commit row at the bottom" />
|
||
</Frame>
|
||
|
||
The panel groups four regions:
|
||
|
||
- **Pending update banner.** Appears at the top when a webhook in **Review only** mode has fetched a new commit. Click **Review** to re-fetch the incoming commit and open the diff dialog.
|
||
- **Form fields.** Repository URL, branch, the ordered compose-file picker, an optional project directory, optional sibling `.env` sync, authentication toggle, and the apply behavior radio group.
|
||
- **Last applied stat strip.** Shows the short SHA of the last commit Sencho applied to disk, plus the timestamp of the most recent successful save or pull.
|
||
- **Footer actions.** **Remove** disconnects the source by exporting the effective compose model into a single `compose.yaml` and removing auto-discovered override files; the remaining materialized files are kept. **Pull now** fetches the configured branch's HEAD; **Save** or **Update** persists form changes after a reachability check passes.
|
||
|
||
## Create a stack from a Git repository
|
||
|
||
Skip the "empty stack then link later" detour and point at a repo from the start. Click **Create Stack** in the sidebar, switch to the **From Git** tab, and fill in the same fields you would on the Git Source panel.
|
||
|
||
<Frame>
|
||
<img src="/images/git-sources/create-from-git-tab.png" alt="New stack dialog with the From Git tab selected, showing stack name, repository URL, branch, compose path, sibling .env toggle, authentication toggle, apply behavior radio group, a Deploy after create checkbox, and an HTTPS REPOS ONLY footer hint" />
|
||
</Frame>
|
||
|
||
Sencho fetches the compose files, validates the merged result with `docker compose config`, writes them to a fresh stack directory, and links the Git source in one step. The last-applied commit SHA is seeded from the fetch so the first manual pull produces a clean diff rather than a "local edits detected" warning.
|
||
|
||
Tick **Deploy after create** to run `docker compose up -d` immediately after the files land. If the deploy fails, the stack and Git source are kept on disk so you can fix the underlying issue (missing image, port conflict, host resources) and retry the deploy from the editor.
|
||
|
||
### Failure modes on create
|
||
|
||
| Situation | What happens |
|
||
|-----------|--------------|
|
||
| Stack name already exists | Sencho returns **409** with "Stack already exists" and makes no changes on disk or in the database. Pick a different name or remove the existing stack. |
|
||
| Repository unreachable or auth failed | Fetch fails before anything is created. The form stays open with an error toast describing the cause. |
|
||
| Fetched compose fails validation | The stack directory is not created and no Git source row is inserted. The error toast shows the `docker compose config` message. |
|
||
| Fetch + validate succeed but optional deploy fails | The stack and Git source are kept. The toast reads "Stack created, but deploy failed: ..." and you can retry the deploy from the editor. |
|
||
|
||
## Configure a source
|
||
|
||
| Field | Description |
|
||
|-------|-------------|
|
||
| **Repository URL** | `https://github.com/your-org/your-repo.git` (HTTPS only) |
|
||
| **Branch** | Branch to track (e.g. `main`) |
|
||
| **Compose files** | One or more paths within the repo, merged in the listed order (e.g. `deploy/base.yaml` then `deploy/prod.yaml`). The first file is the primary. Reorder by dragging (or the up/down arrows on a phone) and remove with the **×** button. |
|
||
| **Project directory** | Optional path within the repo passed to `docker compose --project-directory`, so relative build contexts, bind mounts, and `env_file` references resolve from that base. Leave blank to use the stack root. |
|
||
| **Also sync sibling `.env` file** | When enabled, also pulls the `.env` from the same directory as the primary compose file. The form shows the resolved path inline (e.g. `deploy/.env` for a primary at `deploy/compose.yaml`). |
|
||
| **Authentication** | **Public (no auth)** for public repos, **Personal Access Token** for private repos |
|
||
| **Apply behavior** | See the three modes below |
|
||
|
||
Saving runs a reachability check against the repository. If the URL is wrong, the token is invalid, the branch does not exist, or a file is missing, Sencho surfaces the error inline and nothing is persisted.
|
||
|
||
### Multiple compose files
|
||
|
||
Many projects split a stack into a base file plus environment overrides. List the files in the order you want them merged and Sencho deploys with `docker compose -f base.yaml -f prod.yaml ...`, applying each later file's values on top of the earlier ones, exactly as the Compose CLI does. The same ordered set is used for every action on the stack: deploy, update, restart, stop, and teardown.
|
||
|
||
On disk, the primary file lands as the stack's `compose.yaml` and each additional file keeps its repository-relative path inside the stack directory, so you can still browse and edit them in the file explorer. A single-file source behaves exactly as before.
|
||
|
||
### Apply behavior modes
|
||
|
||
| Mode | What happens when a webhook fires |
|
||
|------|-----------------------------------|
|
||
| **Review only** | Sencho fetches and validates the incoming commit and marks the stack as having a pending update. You review the diff and apply manually. |
|
||
| **Auto-write files** | Sencho writes the new compose and env to disk automatically but does not redeploy. Use this when another process handles rollout. |
|
||
| **Auto-deploy** | Sencho writes the files and immediately runs `docker compose up -d` so the stack picks up the new configuration. |
|
||
|
||
Auto-deploy implies Auto-write: you cannot deploy automatically without also writing the new files first.
|
||
|
||
You can always override on the spot: when you click **Apply** in the diff dialog, a **Deploy after apply** checkbox lets you deploy regardless of the configured mode.
|
||
|
||
## Pulling and reviewing changes
|
||
|
||
Click **Pull now** on the Git Source panel to fetch the latest commit on the configured branch.
|
||
|
||
<Frame>
|
||
<img src="/images/git-sources/diff-dialog.png" alt="GIT · PULL PREVIEW dialog for the demo-app stack, showing a Local edits detected on disk warning above a Monaco side-by-side diff between the on-disk compose.yaml and the incoming commit, with a Deploy after apply checkbox and an Apply button in the footer" />
|
||
</Frame>
|
||
|
||
The diff dialog shows:
|
||
|
||
- The `GIT · PULL PREVIEW` kicker and the short SHA of the incoming commit at the top.
|
||
- A side-by-side compare of the on-disk compose file and the incoming version.
|
||
- A `.env` tab when the source is configured to sync `.env`.
|
||
- An **Incoming compose failed validation** banner when the incoming compose fails `docker compose config`. The Apply button stays disabled until validation passes.
|
||
- A **Local edits detected on disk** banner when the on-disk content differs from the last applied commit. Applying in this state opens an **Overwrite local edits?** confirmation modal whose primary button is **Overwrite and apply**.
|
||
|
||
### Pending updates
|
||
|
||
When a webhook fires in **Review only** mode, the stack gets a pending GitBranch icon next to its row in the sidebar and a pulsing dot on the **Git Source** button in the editor. Clicking either re-fetches the commit and opens the diff dialog; the panel also shows a **Pending update** banner with a **Review** button.
|
||
|
||
If the same stack also has an image update available, the image-update dot in the sidebar takes priority over the Git source icon, so only the update dot renders. The pending Git source is still surfaced inside the editor on the **Git Source** button.
|
||
|
||
<Frame>
|
||
<img src="/images/git-sources/sidebar-badge.png" alt="Sidebar stack list with a small GitBranch icon next to the demo-app entry indicating a pending Git source update" />
|
||
</Frame>
|
||
|
||
Click **Dismiss** in the diff dialog to discard a pending update without applying.
|
||
|
||
## Trigger from CI with a webhook
|
||
|
||
Git sources integrate with Sencho's existing webhook system. Create a webhook targeting the stack with the **Git source sync** action.
|
||
|
||
<Frame>
|
||
<img src="/images/git-sources/webhook-action.png" alt="New webhook form with Name, Stack and Action fields. The Action select is open, showing Deploy, Restart, Stop, Start, Pull and Update, and Git source sync as options, with Git source sync highlighted at the bottom" />
|
||
</Frame>
|
||
|
||
The webhook's behavior on trigger depends on the source's apply mode:
|
||
|
||
- **Review only**: fetch, validate, diff, mark pending.
|
||
- **Auto-write files**: fetch, validate, write to disk.
|
||
- **Auto-deploy**: fetch, validate, write, deploy.
|
||
|
||
The Git source sync action is only selectable on webhooks whose target stack already has a Git source configured. Webhook triggers for a single source are debounced on a 10-second window so a runaway pipeline cannot overwhelm Sencho (or your repository host's rate limits); the dashboard records the skipped trigger in the webhook's execution history.
|
||
|
||
### GitHub Actions example
|
||
|
||
```yaml
|
||
- name: Sync compose via Sencho
|
||
run: |
|
||
BODY='{}'
|
||
SIGNATURE=$(echo -n "$BODY" | openssl dgst -sha256 -hmac "${{ secrets.SENCHO_WEBHOOK_SECRET }}" | cut -d' ' -f2)
|
||
curl -X POST "${{ secrets.SENCHO_URL }}/api/webhooks/${{ secrets.SENCHO_WEBHOOK_ID }}/trigger" \
|
||
-H "Content-Type: application/json" \
|
||
-H "X-Webhook-Signature: sha256=$SIGNATURE" \
|
||
-d "$BODY"
|
||
```
|
||
|
||
See the [Webhooks](/features/webhooks) page for the full signing protocol.
|
||
|
||
## Private repositories
|
||
|
||
For private repositories, use a Personal Access Token scoped to read access on the target repo:
|
||
|
||
- **GitHub**: a fine-grained PAT with **Contents: Read** permission on the repo, or a classic PAT with the `repo` scope.
|
||
- **GitLab**: a project or group access token with the `read_repository` scope.
|
||
- **Bitbucket**: an app password with **Repositories: Read**.
|
||
|
||
Paste the token into the **Token** field and save. Sencho stores it encrypted at rest and never returns it in API responses or UI. When editing the source later, the token field shows a masked placeholder; leave it blank to keep the stored value, or type a new token to replace it. Switching the auth type back to **Public (no auth)** clears the stored token.
|
||
|
||
The encryption boundary covers the pending update payload too: every pull caches the fetched compose and env content in the database so the diff dialog can reopen without a refetch, and that cached content is encrypted at rest in the same way as the token, since compose files routinely embed secrets via env interpolation.
|
||
|
||
## Local edits vs Git
|
||
|
||
Sencho tracks a hash of the compose and env contents at the moment of the last apply. When you pull, it compares that hash against the current on-disk content.
|
||
|
||
- Matching hash: applying overwrites content that Sencho itself last wrote.
|
||
- Differing hash: someone edited the files outside Git. The diff dialog shows the **Local edits detected on disk** banner, and Apply requires the **Overwrite local edits?** confirmation.
|
||
|
||
The in-browser editor and the Git Source panel both write to the same files, so you can always fall back to editing locally. The next pull will just flag the divergence rather than silently clobbering your edits.
|
||
|
||
Pulls, applies, and create-from-git operations on the same stack are serialized by a per-stack lock, so a webhook that fires in the middle of a manual apply waits for the apply to finish rather than racing it.
|
||
|
||
## Troubleshooting
|
||
|
||
<AccordionGroup>
|
||
<Accordion title="Repository not found or not accessible">
|
||
Verify the URL is reachable from the Sencho host and ends with `.git`. GitHub returns a "not found" response for both genuinely missing repos and private repos you cannot read, so Sencho tailors the hint based on what you provided:
|
||
|
||
- **No token configured**: the repo might be private. Switch **Authentication** to **Personal Access Token** and paste a token with read access.
|
||
- **Token configured**: double-check the URL is correct and the token has read access to this specific repo. GitHub fine-grained PATs need **Contents: Read** on the target repo; classic PATs need the `repo` scope.
|
||
</Accordion>
|
||
|
||
<Accordion title="Authentication failed">
|
||
You supplied a token and the Git host rejected it outright. The token is missing, expired, or lacks read access. Generate a new token and replace the value in the **Token** field. Sencho returns this as a 400 form error rather than a 401, so an upstream auth failure does not sign you out of the dashboard.
|
||
</Accordion>
|
||
|
||
<Accordion title="Branch not found">
|
||
The branch name is case-sensitive and must exist on the remote. Confirm the branch with `git ls-remote <url>` from a shell that has access.
|
||
</Accordion>
|
||
|
||
<Accordion title="File not found">
|
||
Each compose path is relative to the repository root and must point at the file, not its parent directory. Every file in the list must exist on the tracked branch; if any is missing, the save or pull fails and names the path. If a file was moved, update its path on the panel and save.
|
||
</Accordion>
|
||
|
||
<Accordion title="Compose validation failed">
|
||
Sencho runs `docker compose config` against the incoming content before letting you apply. The error banner shows the exact message. Common causes: unresolved `${VAR}` interpolation (commit a `.env` file next to the compose file and enable sibling `.env` sync), invalid `include:` paths, or schema issues introduced by a recent compose change. Validation has a 10-second budget; an unusually large compose with many services may need to be split.
|
||
</Accordion>
|
||
|
||
<Accordion title="Local edits detected">
|
||
The on-disk files diverge from the last applied Git commit. Either confirm **Overwrite and apply** to take the incoming content, or discard local work with a redeploy from the stack editor, or commit your local changes back to the repo so the diff becomes clean.
|
||
</Accordion>
|
||
|
||
<Accordion title="Webhook skipped (rate limited)">
|
||
Sencho debounces rapid-fire triggers on a 10-second window per source. Wait at least 10 seconds and retry, or consolidate multiple CI triggers into a single call at the end of your pipeline. Skipped triggers appear in the webhook's execution history.
|
||
</Accordion>
|
||
|
||
<Accordion title="Network timeout">
|
||
The clone did not finish in time. Fetches run with a 30-second timeout to keep a slow or unreachable host from hanging the stack panel. Check that the Sencho host can reach the repository host (proxies, firewalls, DNS) and try again. If the repository is genuinely large, pin a smaller compose subpath or mirror it somewhere closer to the Sencho host.
|
||
</Accordion>
|
||
|
||
<Accordion title="Repository too large">
|
||
A clone is capped on how much it downloads (100 MB by default), and individual compose/env files are capped on read. A compose repository is normally tiny, so hitting either usually means the tracked branch carries large binaries. Point the source at a repository or branch that holds just your compose and `.env` files. If a large repository is unavoidable, an operator can raise the download ceiling with the `GITSOURCE_MAX_CLONE_BYTES` environment variable.
|
||
</Accordion>
|
||
|
||
<Accordion title="Pending commit has changed since this pull was fetched">
|
||
You opened a diff dialog, then a webhook fired and replaced the pending commit before you clicked **Apply**. Close the dialog and reopen the panel to load the latest pending commit; the **Review** button will fetch the newer one.
|
||
</Accordion>
|
||
|
||
<Accordion title="Applied but deploy failed">
|
||
The incoming compose file was written to disk successfully, but the subsequent `docker compose up -d` did not complete. The toast message includes the underlying reason (for example, an image pull failure or a port conflict). The stack is already on the new content, so you can retry the deploy directly from the editor's **Deploy** button without re-pulling. Fix the root cause first (image availability, host resources, network config) and redeploy.
|
||
</Accordion>
|
||
|
||
<Accordion title="Stack contents look wrong or appear empty">
|
||
If the compose file in your repository is tracked via Git LFS, Sencho will refuse the link and surface an LFS error rather than write a pointer stub as real content. Commit the plain compose file (and any synced `.env`) without LFS, or replace the LFS pointer in-place, then retry.
|
||
</Accordion>
|
||
|
||
<Accordion title="A build context or volume points at an empty folder">
|
||
Repositories that use Git submodules do not have their submodule contents cloned during a Git Source fetch. Sencho surfaces a warning when `.gitmodules` is present, and refuses inputs and build contexts that reference submodule contents with an actionable message. Inline the referenced files into the main repository or flatten the submodule so the paths resolve at deploy time.
|
||
</Accordion>
|
||
|
||
<Accordion title="An additional file named compose.yaml is rejected">
|
||
The first file in the list is always written to the stack's root `compose.yaml`. To avoid clobbering it, an additional file whose repository path is also `compose.yaml` is rejected. Rename it in the repository, or move it to the top of the list to make it the primary.
|
||
</Accordion>
|
||
|
||
<Accordion title="A relative build context or extends target is missing with multiple files">
|
||
Git Sources materializes the complete project, including recursive `include:` and `extends.file` dependencies, service env files, file-backed configs and secrets, and build contexts. If a referenced file is still missing, the pull refused it and the refusal message names the path and the reason (out-of-bound path, Git LFS pointer, submodule, symlink, or a size cap). Fix the declaration in the repository and pull again.
|
||
</Accordion>
|
||
|
||
<Accordion title="Only HTTPS is supported">
|
||
Git Sources fetch over HTTPS only. SSH clone URLs (`git@host:org/repo.git`) and custom protocols are rejected with a client-side validation error. Paste the `https://...` URL and use a Personal Access Token for authentication on private repositories.
|
||
</Accordion>
|
||
</AccordionGroup>
|
||
|
||
## Known limitations
|
||
|
||
- **HTTPS only.** SSH URLs and SSH keys are not supported. Use a Personal Access Token for private repos.
|
||
- **No Git LFS.** Compose and env files stored via LFS are rejected. Commit plain files instead.
|
||
- **No submodules.** Submodule contents are not fetched. Inputs and build contexts that reference submodule contents are refused with an actionable message; a warning is shown when `.gitmodules` is present.
|
||
- **Branch-tracking only.** Sources follow the head of a branch. Specific commit SHAs and tags are not pinnable. Each pull resolves and pins the exact commit SHA, so apply always materializes the reviewed revision.
|
||
- **Clone size cap.** A clone is bounded on how much it downloads (and each compose/env file is capped on read), so very large repositories are rejected. Operators can adjust the download ceiling with `GITSOURCE_MAX_CLONE_BYTES`.
|
||
- **Complete project materialization.** Every repository-local input the project needs is materialized: the ordered compose files, implicit `compose.override.*` files, recursive `include:` and `extends.file` dependencies, service env files, file-backed configs and secrets, label files, and build contexts with `.dockerignore` semantics. The materialized set is recorded in a versioned managed-project manifest, and each pull stages a candidate that is validated with the exact deployment invocation before anything on disk changes. If apply is interrupted, Sencho completes the accepted generation or restores the previous generation. If files were edited during the interruption and no longer match either generation, Sencho preserves them and requires manual recovery instead of overwriting them.
|
||
- **Unsupported inputs are refused, not guessed.** Inputs that cannot be safely reproduced fail the pull with an actionable message: URL includes, Git LFS pointers, submodule contents, symbolic links, build contexts that exceed the size bounds, and include or extends declarations that point outside the repository or use dynamic `\${VAR}` paths (their contents cannot be enumerated). Nothing is applied until the declaration is fixed. Absolute host paths, host bind mounts, external resources, and dynamic `\${VAR}` data paths are never claimed as covered: they resolve at deploy time from the environment or the node, and the manifest records them as unmanaged.
|
||
- **Materialization bounds.** The materialized project is bounded by file count, total bytes, per-file size, path depth, and build-context size, each adjustable with a `GITSOURCE_*` variable (see configuration). Crossing a bound refuses the pull with the counts so far rather than producing a partial project.
|
||
- **Detach and export.** Removing a Git source renders the effective compose model into a single `compose.yaml`, keeps the remaining materialized files, removes auto-discovered override files so the exported model is final, and removes Git tracking. Resolved environment values are baked into the exported file. If removal is interrupted before it completes, Sencho restores the original files automatically.
|
||
- **Rollback scope.** Rollback of a Git-managed stack restores compose files and `.env`. Other materialized inputs are not reverted by rollback; re-apply the previous revision from Git to restore them.
|
||
- **Some read-only views read the primary file.** The dependency graph, drift snapshot, and networking inspector summarize the primary compose file, so a service declared only in an override may not appear in those views. Deploy, update, image-update checks, and mesh attachment use the full merged set.
|