mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-30 20:29:15 +00:00
feat(gitops): resolve branch, tag, and SHA refs to immutable commits before fetch (#1864)
* feat(gitops): resolve branch, tag, and SHA refs to immutable commits before fetch The ref model now resolves a configured branch, tag, or full commit SHA to an immutable commit before any content is downloaded, and records both the configured and the resolved identity where revision state persists. - RefKind (branch | tag | sha) is a resolved property, not caller-asserted. A bare string resolves branch-first, then tag; a full 40/64-hex SHA self-resolves with no remote round-trip. Branch and tag both fetch via a bare --branch name; a SHA uses init + shallow fetch + detached checkout. - A single ls-remote with narrow heads/tags refspecs pins the configured ref to an immutable SHA; rev-parse HEAD must equal the resolved SHA or the fetch refuses (tip-changed) instead of materializing unreviewed content. - Error union grew: REF_NOT_FOUND (ref-neutral, replaces BRANCH_NOT_FOUND), UNSUPPORTED_REF (a pinned SHA the host will not serve), and a service-level REF_DELETED upgrade that fires when a classified REF_NOT_FOUND occurs for a source with prior fetch history (a vanished ref reads as delete/force-push, not a fresh typo). Status mapping: REF_NOT_FOUND/REF_DELETED to 404, UNSUPPORTED_REF to 400. - Configured-vs-resolved identity is recorded via a nullable resolved_ref_kind column on gitops_generations (added to CREATE TABLE and re-added for legacy installs through maybeAddCol). The kind is deliberately NOT in the plan fingerprint: two sources naming the same commit differently are the same plan. Docs updated (git-sources feature page, connect-a-git-source tutorial, and the native-git-transport internal deep-dive) to the ref-neutral naming. * fix(gitops): harden ref resolution after pre-merge audit Request peeled annotated-tag refs from ls-remote, detect force-pushes and ref-kind changes against prior fetch identity, persist resolved kind on application rows, and add real-git tag/SHA integration coverage plus ref-neutral UI and operator docs. * test(gitops): mock verifyFastForward in direct producer suite The producer tests stub the transport seam but were missing resolved kind on resolveRef and a verifyFastForward stub, so second pulls tripped the new ref-continuity checks as REF_DELETED. * test(git): remove unused buildBareFixtureRepo helper Fixes backend lint failure after the integration fixture was refactored to buildRichFixtureRepo without dropping the old wrapper. * fix(gitops): correct fast-forward ancestry verification under size bounds Replace the dual shallow-fetch ancestry probe with a single-tip deepen strategy, keep verifier Git work inside the transport watchdog, and add real-Git regression coverage for linear advances and rewritten history. * fix(gitops): bound fast-forward verification with exponential deepen Replace per-commit deepen loops with exponential steps, cap remote fetch rounds, and share one deadline across verifier Git calls. Budget exhaustion now surfaces as a classified timeout instead of REF_DELETED. * fix(gitops): classify fast-forward probe failures accurately Normalize verifier probe timeouts and unexpected exit codes into transport failures, interpret merge-base status 1 as proven non-ancestry only, and treat shallow stagnation as timeout instead of REF_DELETED. * fix(gitops): satisfy tsc on probeFailure never returns * fix(gitops): address Phase E QA findings on ref verification Remove the fast-forward scratch repo after verification so pull size caps are not inflated, classify GitHub not-our-ref as UNSUPPORTED_REF, persist fetched_resolved_ref_kind on create-from-git, and broaden REF_DELETED copy for retagged tags.
This commit is contained in:
Anso
@@ -3,7 +3,7 @@ title: Git Sources
|
||||
description: Link a stack to a Git repository and keep one or more compose files in sync via manual pulls or webhook triggers.
|
||||
---
|
||||
|
||||
Git Sources turn any stack into a GitOps target. Point Sencho at a repository and branch, choose one or more compose files to merge in order, pull updates on demand or from CI, and review a classified change plan before applying files to disk. Optional sibling `.env` sync keeps configuration consistent too.
|
||||
Git Sources turn any stack into a GitOps target. Point Sencho at a repository and ref (branch, tag, or commit SHA), choose one or more compose files to merge in order, pull updates on demand or from CI, and review a classified change plan before applying files to disk. Optional sibling `.env` sync keeps configuration consistent too.
|
||||
|
||||
<Note>
|
||||
Git Sources are available on every tier, including Community.
|
||||
@@ -12,7 +12,7 @@ Git Sources turn any stack into a GitOps target. Point Sencho at a repository an
|
||||
## How it works
|
||||
|
||||
1. Open a stack and click the **Git Source** button in the editor toolbar.
|
||||
2. Fill in the repository URL and branch, then choose the compose files. Use **Browse** to pick them from the repository tree, or type a path and press Enter. Add a token if the repo is private.
|
||||
2. Fill in the repository URL and ref, then choose the compose files. Use **Browse** to pick them from the repository tree, or type a path and press Enter. Add a token if the repo is private.
|
||||
3. Click **Pull now** to fetch the latest commit. Sencho opens a classified change plan: adds, modifications, removals, and any local conflicts.
|
||||
4. Click **Apply** to write the incoming files to disk. Apply stays disabled while local file conflicts are present. A Compose invocation change (for example a `.env` file added or removed outside Git) is shown in the plan and does not disable Apply. Applying records the incoming invocation as the new baseline and leaves unmanaged files on disk. Tick **Deploy after apply** in the same dialog to redeploy in one step.
|
||||
|
||||
@@ -21,15 +21,15 @@ Writes land in the stack's existing directory using the same storage Sencho uses
|
||||
## Anatomy of the panel
|
||||
|
||||
<Frame>
|
||||
<img src="/images/git-sources/panel.png" alt="Git Source panel for a stack already linked to a repository, showing populated Repository URL, Branch, Compose file path, the Authentication toggle, the Apply behavior radio group, and a Last applied commit row at the bottom" />
|
||||
<img src="/images/git-sources/panel.png" alt="Git Source panel for a stack already linked to a repository, showing populated Repository URL, Ref, Compose file path, the Authentication toggle, the Apply behavior radio group, and a Last applied commit row at the bottom" />
|
||||
</Frame>
|
||||
|
||||
The panel groups four regions:
|
||||
|
||||
- **Pending update banner.** Appears at the top whenever a fetched commit is staged, however it was fetched. Its heading is the source state, so it says whether the commit is ready to apply, waiting on review, or blocked by local conflicts. Click **Review** to re-fetch the incoming commit and open the change plan.
|
||||
- **Form fields.** Repository URL, branch, the ordered compose-file picker, an optional project directory, optional sibling `.env` sync, authentication toggle, and the apply behavior radio group.
|
||||
- **Form fields.** Repository URL, ref, the ordered compose-file picker, an optional project directory, optional sibling `.env` sync, authentication toggle, and the apply behavior radio group.
|
||||
- **Last applied stat strip.** Shows the short SHA of the last commit Sencho applied to disk, the source state (see below), and the timestamp of the most recent successful save or pull.
|
||||
- **Footer actions.** **Remove** disconnects the source by exporting the effective compose model into a single `compose.yaml` and removing auto-discovered override files; the remaining materialized files are kept. **Pull now** fetches the configured branch's HEAD; **Save** or **Update** persists form changes after a reachability check passes.
|
||||
- **Footer actions.** **Remove** disconnects the source by exporting the effective compose model into a single `compose.yaml` and removing auto-discovered override files; the remaining materialized files are kept. **Pull now** fetches the configured branch, tag, or commit's current revision; **Save** or **Update** persists form changes after a reachability check passes.
|
||||
|
||||
## Source state
|
||||
|
||||
@@ -55,7 +55,7 @@ These are the states you will see most often.
|
||||
The last two matter most. Sencho reports an interrupted operation as unknown rather than guessing, so a stack whose pull was cut short by a restart says so instead of quietly reading as up to date.
|
||||
|
||||
<Note>
|
||||
Changing the repository, the branch, the compose files, the project directory, or the `.env` sync clears any staged commit, because the plan was built against the settings you just replaced. Pull again to rebuild it. The state moves to **Reconcile required** if a commit had already been accepted, and to **Never reconciled** if none ever was. Changing only the token or the apply behavior leaves a staged commit alone, since neither changes what would be materialized.
|
||||
Changing the repository, the ref, the compose files, the project directory, or the `.env` sync clears any staged commit, because the plan was built against the settings you just replaced. Pull again to rebuild it. The state moves to **Reconcile required** if a commit had already been accepted, and to **Never reconciled** if none ever was. Changing only the token or the apply behavior leaves a staged commit alone, since neither changes what would be materialized.
|
||||
</Note>
|
||||
|
||||
### When part of the state could not be proven
|
||||
@@ -90,14 +90,14 @@ Tick **Deploy after create** to run `docker compose up -d` immediately after the
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| **Repository URL** | `https://github.com/your-org/your-repo.git` (HTTPS only) |
|
||||
| **Branch** | Branch to track (e.g. `main`) |
|
||||
| **Ref** | Branch, tag, or full commit SHA to track (e.g. `main`, `v1.0`, or a 40-character SHA) |
|
||||
| **Compose files** | One or more paths within the repo, merged in the listed order (e.g. `deploy/base.yaml` then `deploy/prod.yaml`). The first file is the primary. Reorder by dragging (or the up/down arrows on a phone) and remove with the **×** button. |
|
||||
| **Project directory** | Optional path within the repo passed to `docker compose --project-directory`, so relative build contexts, bind mounts, and `env_file` references resolve from that base. Leave blank to use the stack root. |
|
||||
| **Also sync sibling `.env` file** | When enabled, also pulls the `.env` from the same directory as the primary compose file. The form shows the resolved path inline (e.g. `deploy/.env` for a primary at `deploy/compose.yaml`). |
|
||||
| **Authentication** | **Public (no auth)** for public repos, **Personal Access Token** for private repos |
|
||||
| **Apply behavior** | See the three modes below |
|
||||
|
||||
Saving runs a reachability check against the repository. If the URL is wrong, the token is invalid, the branch does not exist, or a file is missing, Sencho surfaces the error inline and nothing is persisted.
|
||||
Saving runs a reachability check against the repository. If the URL is wrong, the token is invalid, the ref does not exist, or a file is missing, Sencho surfaces the error inline and nothing is persisted.
|
||||
|
||||
### Multiple compose files
|
||||
|
||||
@@ -119,7 +119,7 @@ You can always override on the spot: when you click **Apply** in the change plan
|
||||
|
||||
## Pulling and reviewing changes
|
||||
|
||||
Click **Pull now** on the Git Source panel to fetch the latest commit on the configured branch.
|
||||
Click **Pull now** on the Git Source panel to fetch the latest commit on the configured branch, tag, or commit.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/git-sources/diff-dialog.png" alt="GIT · CHANGE PLAN dialog for the demo-app stack, listing classified file operations (add, modify, remove) with a Deploy after apply checkbox and an Apply button in the footer" />
|
||||
@@ -217,8 +217,16 @@ Pulls, applies, and create-from-git operations on the same stack are serialized
|
||||
You supplied a token and the Git host rejected it outright. The token is missing, expired, or lacks read access. Generate a new token and replace the value in the **Token** field. Sencho returns this as a 400 form error rather than a 401, so an upstream auth failure does not sign you out of the dashboard.
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Branch not found">
|
||||
The branch name is case-sensitive and must exist on the remote. Confirm the branch with `git ls-remote <url>` from a shell that has access.
|
||||
<Accordion title="Branch or tag not found">
|
||||
The configured branch or tag is case-sensitive and must exist on the remote. Confirm it with `git ls-remote <url>` from a shell that has access. Sencho resolves the ref before fetching, so a typo or a ref that was never pushed surfaces here rather than as a generic failure. A full commit SHA never produces this error: it resolves to itself, so a SHA the host refuses to serve reports as a host-capability problem under "Commit not reachable on this host".
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Branch or tag deleted or force-pushed">
|
||||
The configured ref previously resolved to a commit but no longer matches that history. The ref may have been deleted, renamed, superseded by a same-named tag, or force-pushed to a history the old commit is no longer part of. Point the source at a current branch, tag, or commit, or restore the ref upstream, then save again.
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Commit not reachable on this host">
|
||||
The configured commit SHA is not one the Git host will serve. Hosts only fetch SHAs they advertise by default, so a commit that is not on any branch or tag tip, or one on a host that blocks unadvertised object fetch, returns this. Use a branch or tag, or a commit the host advertises.
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="File not found">
|
||||
@@ -283,7 +291,7 @@ Pulls, applies, and create-from-git operations on the same stack are serialized
|
||||
- **HTTPS only.** SSH URLs and SSH keys are not supported. Use a Personal Access Token for private repos.
|
||||
- **No Git LFS.** Compose and env files stored via LFS are rejected. Commit plain files instead.
|
||||
- **No submodules.** Submodule contents are not fetched. Inputs and build contexts that reference submodule contents are refused with an actionable message; a warning is shown when `.gitmodules` is present.
|
||||
- **Branch-tracking only.** Sources follow the head of a branch. Specific commit SHAs and tags are not pinnable. Each pull resolves and pins the exact commit SHA, so apply always materializes the reviewed revision.
|
||||
- **Refs, not arbitrary commits.** Sources follow a branch head, a tag, or a pinned commit SHA. Each pull resolves the configured ref to the exact commit it currently points at and pins that SHA, so apply always materializes the reviewed revision. A commit SHA not advertised by the Git host is refused.
|
||||
- **Clone size cap.** A clone is bounded on the on-disk size of its temporary workspace (and each compose/env file is capped on read), so very large repositories are rejected. Operators can adjust the workspace ceiling with `GITSOURCE_MAX_CLONE_BYTES`.
|
||||
- **Complete project materialization.** Every repository-local input the project needs is materialized: the ordered compose files, implicit `compose.override.*` files, recursive `include:` and `extends.file` dependencies, service env files, file-backed configs and secrets, label files, and build contexts with `.dockerignore` semantics. The materialized set is recorded in a versioned managed-project manifest, and each pull stages a candidate that is validated with the exact deployment invocation before anything on disk changes. If apply is interrupted, Sencho completes the accepted generation or restores the previous generation. If files were edited during the interruption and no longer match either generation, Sencho preserves them and requires manual recovery instead of overwriting them.
|
||||
- **Unsupported inputs are refused, not guessed.** Inputs that cannot be safely reproduced fail the pull with an actionable message: URL includes, Git LFS pointers, submodule contents, symbolic links, build contexts that exceed the size bounds, and include or extends declarations that point outside the repository or use dynamic `\${VAR}` paths (their contents cannot be enumerated). Nothing is applied until the declaration is fixed. Absolute host paths, host bind mounts, external resources, and dynamic `\${VAR}` data paths are never claimed as covered: they resolve at deploy time from the environment or the node, and the manifest records them as unmanaged.
|
||||
|
||||
@@ -31,11 +31,11 @@ This tutorial covers linking an existing stack to a Git source and running one m
|
||||
Open the `marketing-site` stack and select **Git Source** in the editor toolbar. The panel opens empty, since nothing is linked yet.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/tutorials/connect-a-git-source/git-source-panel-empty.png" alt="Empty Git source panel for the marketing-site stack, showing blank Repository URL and Branch fields defaulted to main, a compose.yaml entry marked primary in Compose files, Public (no auth) selected under Authentication, and Review only selected under Apply behavior." />
|
||||
<img src="/images/tutorials/connect-a-git-source/git-source-panel-empty.png" alt="Empty Git source panel for the marketing-site stack, showing blank Repository URL and Ref fields defaulted to main, a compose.yaml entry marked primary in Compose files, Public (no auth) selected under Authentication, and Review only selected under Apply behavior." />
|
||||
</Frame>
|
||||
</Step>
|
||||
<Step title="Point it at your repository">
|
||||
Paste your repository's HTTPS URL into **Repository URL** (for example `https://github.com/your-org/your-repo.git`). Leave **Branch** on its default, `main`, unless your repository uses a different one.
|
||||
Paste your repository's HTTPS URL into **Repository URL** (for example `https://github.com/your-org/your-repo.git`). Leave **Ref** on its default, `main`, unless your repository uses a different branch, tag, or pinned commit SHA.
|
||||
|
||||
Select **Browse** to confirm Sencho can actually reach the repository and see its files, rather than trusting the URL is correct. The browser lists every file in the repository; `compose.yaml` is already ticked as the primary compose file, since that name matches the picker's default.
|
||||
|
||||
@@ -46,9 +46,9 @@ This tutorial covers linking an existing stack to a Git source and running one m
|
||||
Leave **Authentication** on **Public (no auth)** for a public repository, and **Apply behavior** on **Review only**, the safest default: a pull only stages a change plan for you to review, it never writes or deploys on its own. Select **Save**. Sencho runs a reachability check against the repository before persisting anything; if that check fails, nothing is saved and the panel reports why.
|
||||
</Step>
|
||||
<Step title="Pull the latest commit">
|
||||
Now make a change the way your team actually would: edit the compose file in your repository (not in Sencho) and push a commit. For this tutorial, bump the pinned tag from `nginx:1.27-alpine` to `nginx:1.28-alpine` and push it to the branch you configured.
|
||||
Now make a change the way your team actually would: edit the compose file in your repository (not in Sencho) and push a commit. For this tutorial, bump the pinned tag from `nginx:1.27-alpine` to `nginx:1.28-alpine` and push it to the ref you configured.
|
||||
|
||||
Back in the Git Source panel, select **Pull now**. Sencho fetches the branch's current commit and opens a classified change plan against what's on disk.
|
||||
Back in the Git Source panel, select **Pull now**. Sencho fetches the ref's current commit and opens a classified change plan against what's on disk.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/tutorials/connect-a-git-source/pull-preview-diff.png" alt="GIT · CHANGE PLAN dialog for marketing-site, listing a Modify row for compose.yaml (image tag change) with Deploy after apply and Apply in the footer." />
|
||||
@@ -81,10 +81,10 @@ Check from two places, since neither alone proves the pull was actually applied
|
||||
|
||||
## If something goes wrong
|
||||
|
||||
**Saving the source fails with "Repository not found or not accessible," even though the URL is right.** Branch names are case-sensitive, and a typo there (`Main` instead of `main`, for example) surfaces as this same repository-level error rather than a distinct branch error, since Sencho can't always tell a missing branch apart from a missing repository during the reachability check. Double-check the branch name against what your Git host actually shows before assuming the URL itself is wrong.
|
||||
**Saving the source fails with "The configured branch, tag, or commit was not found," even though the URL is right.** Ref names are case-sensitive, and a typo there (`Main` instead of `main`, for example) surfaces as a distinct not-found error rather than a repository error, because Sencho resolves the ref on the remote before fetching. Double-check the ref against what your Git host actually shows before assuming the URL itself is wrong.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/tutorials/connect-a-git-source/branch-error.png" alt="Git source panel with Branch set to the incorrect value Main, and a red error toast in the corner reading Repository not found or not accessible." />
|
||||
<img src="/images/tutorials/connect-a-git-source/branch-error.png" alt="Git source panel with Ref set to the incorrect value Main, and a red error toast in the corner reading The configured branch, tag, or commit was not found." />
|
||||
</Frame>
|
||||
|
||||
## Related
|
||||
|
||||
Reference in New Issue
Block a user