feat(gitops): resolve branch, tag, and SHA refs to immutable commits before fetch (#1864)

* feat(gitops): resolve branch, tag, and SHA refs to immutable commits before fetch

The ref model now resolves a configured branch, tag, or full commit SHA to
an immutable commit before any content is downloaded, and records both the
configured and the resolved identity where revision state persists.

- RefKind (branch | tag | sha) is a resolved property, not caller-asserted.
  A bare string resolves branch-first, then tag; a full 40/64-hex SHA
  self-resolves with no remote round-trip. Branch and tag both fetch via a
  bare --branch name; a SHA uses init + shallow fetch + detached checkout.
- A single ls-remote with narrow heads/tags refspecs pins the configured ref
  to an immutable SHA; rev-parse HEAD must equal the resolved SHA or the
  fetch refuses (tip-changed) instead of materializing unreviewed content.
- Error union grew: REF_NOT_FOUND (ref-neutral, replaces BRANCH_NOT_FOUND),
  UNSUPPORTED_REF (a pinned SHA the host will not serve), and a service-level
  REF_DELETED upgrade that fires when a classified REF_NOT_FOUND occurs for a
  source with prior fetch history (a vanished ref reads as delete/force-push,
  not a fresh typo). Status mapping: REF_NOT_FOUND/REF_DELETED to 404,
  UNSUPPORTED_REF to 400.
- Configured-vs-resolved identity is recorded via a nullable resolved_ref_kind
  column on gitops_generations (added to CREATE TABLE and re-added for legacy
  installs through maybeAddCol). The kind is deliberately NOT in the plan
  fingerprint: two sources naming the same commit differently are the same plan.

Docs updated (git-sources feature page, connect-a-git-source tutorial, and the
native-git-transport internal deep-dive) to the ref-neutral naming.

* fix(gitops): harden ref resolution after pre-merge audit

Request peeled annotated-tag refs from ls-remote, detect force-pushes and
ref-kind changes against prior fetch identity, persist resolved kind on
application rows, and add real-git tag/SHA integration coverage plus
ref-neutral UI and operator docs.

* test(gitops): mock verifyFastForward in direct producer suite

The producer tests stub the transport seam but were missing resolved kind
on resolveRef and a verifyFastForward stub, so second pulls tripped the new
ref-continuity checks as REF_DELETED.

* test(git): remove unused buildBareFixtureRepo helper

Fixes backend lint failure after the integration fixture was refactored
to buildRichFixtureRepo without dropping the old wrapper.

* fix(gitops): correct fast-forward ancestry verification under size bounds

Replace the dual shallow-fetch ancestry probe with a single-tip deepen
strategy, keep verifier Git work inside the transport watchdog, and add
real-Git regression coverage for linear advances and rewritten history.

* fix(gitops): bound fast-forward verification with exponential deepen

Replace per-commit deepen loops with exponential steps, cap remote fetch
rounds, and share one deadline across verifier Git calls. Budget exhaustion
now surfaces as a classified timeout instead of REF_DELETED.

* fix(gitops): classify fast-forward probe failures accurately

Normalize verifier probe timeouts and unexpected exit codes into transport
failures, interpret merge-base status 1 as proven non-ancestry only, and
treat shallow stagnation as timeout instead of REF_DELETED.

* fix(gitops): satisfy tsc on probeFailure never returns

* fix(gitops): address Phase E QA findings on ref verification

Remove the fast-forward scratch repo after verification so pull size
caps are not inflated, classify GitHub not-our-ref as UNSUPPORTED_REF,
persist fetched_resolved_ref_kind on create-from-git, and broaden
REF_DELETED copy for retagged tags.
This commit is contained in:
Anso
2026-08-28 20:15:37 +00:00
committed by GitHub
parent 7cd42699d1
commit 48f010475b
38 changed files with 1397 additions and 137 deletions
@@ -19,10 +19,15 @@ describe('gitSourceStatus', () => {
it('maps resource-missing codes to 404', () => {
expect(gitSourceStatus('REPO_NOT_FOUND')).toBe(404);
expect(gitSourceStatus('BRANCH_NOT_FOUND')).toBe(404);
expect(gitSourceStatus('REF_NOT_FOUND')).toBe(404);
expect(gitSourceStatus('REF_DELETED')).toBe(404);
expect(gitSourceStatus('FILE_NOT_FOUND')).toBe(404);
});
it('maps UNSUPPORTED_REF to 400', () => {
expect(gitSourceStatus('UNSUPPORTED_REF')).toBe(400);
});
it('maps NETWORK_TIMEOUT to 504', () => {
expect(gitSourceStatus('NETWORK_TIMEOUT')).toBe(504);
});
@@ -46,6 +46,7 @@ function directApplicationFixture(id: string, stackName: string): GitOpsApplicat
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -6,7 +6,7 @@
* - validateCompose YAML pre-check (empty / non-object / syntax error)
* - Token round-trip via upsert: encryption, has_token projection, undefined/null/empty/non-empty semantics
* - Apply-matrix rejection (auto_deploy requires auto_apply)
* - Error code mapping from native-git transport failures (REPO_NOT_FOUND, AUTH_FAILED, BRANCH_NOT_FOUND, NETWORK_TIMEOUT)
* - Error code mapping from native-git transport failures (REPO_NOT_FOUND, AUTH_FAILED, REF_NOT_FOUND, REF_DELETED, UNSUPPORTED_REF, NETWORK_TIMEOUT)
* - Credential scrubbing in surfaced error messages
* - Pending state lifecycle (setPending -> apply clears -> dismissPending clears)
* - Webhook debounce enforcement
@@ -29,9 +29,10 @@ import {
// ── Hoisted mocks ──────────────────────────────────────────────────────
const { mockResolveRef, mockFetchAtCommit, mockGitClone, mockGitLog } = vi.hoisted(() => ({
const { mockResolveRef, mockFetchAtCommit, mockVerifyFastForward, mockGitClone, mockGitLog } = vi.hoisted(() => ({
mockResolveRef: vi.fn(),
mockFetchAtCommit: vi.fn(),
mockVerifyFastForward: vi.fn(),
mockGitClone: vi.fn(),
mockGitLog: vi.fn(),
}));
@@ -44,6 +45,7 @@ vi.mock('../services/git/nativeGitTransport', () => ({
resolveRef: mockResolveRef,
fetchAtCommit: mockFetchAtCommit,
},
verifyFastForward: mockVerifyFastForward,
}));
@@ -102,6 +104,7 @@ afterAll(() => {
beforeEach(() => {
mockResolveRef.mockReset();
mockFetchAtCommit.mockReset();
mockVerifyFastForward.mockReset();
mockGitClone.mockReset();
mockGitLog.mockReset();
wireTransportDefaults();
@@ -136,10 +139,11 @@ beforeEach(() => {
* at the workspace checkout.
*/
function wireTransportDefaults(): void {
mockVerifyFastForward.mockResolvedValue(true);
mockResolveRef.mockImplementation(async () => {
const log = await mockGitLog({});
const oid = Array.isArray(log) ? log[0]?.oid : undefined;
return { commitSha: oid ?? '' };
return { commitSha: oid ?? '', kind: 'branch' as const, ref: 'main' };
});
mockFetchAtCommit.mockImplementation(async (req: { workspaceRoot: string; commitSha: string }) => {
const path = await import('path');
@@ -708,6 +712,7 @@ describe('GitSourceService error mapping', () => {
expect(mockFetchAtCommit.mock.calls[0][0]).toMatchObject({
commitSha: sha,
ref: 'main',
refKind: 'branch',
repoUrl: 'https://github.com/example/repo.git',
token: 'tok-abc',
workspaceRoot: expect.any(String),
@@ -727,12 +732,12 @@ describe('GitSourceService error mapping', () => {
expect(fsMod.existsSync(failureRoot)).toBe(false);
});
it('reports BRANCH_NOT_FOUND for a branch with no commits', async () => {
// Resolve-first turns an empty branch into a missing remote head.
it('reports REF_NOT_FOUND for a branch with no commits', async () => {
// Resolve-first turns an empty branch into a missing remote ref.
mockGitLog.mockResolvedValue([]);
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({
code: 'BRANCH_NOT_FOUND',
message: expect.stringMatching(/Branch not found/),
code: 'REF_NOT_FOUND',
message: expect.stringMatching(/was not found/),
});
});
@@ -741,12 +746,102 @@ describe('GitSourceService error mapping', () => {
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'REPO_NOT_FOUND' });
});
it('maps remote-branch-not-found to BRANCH_NOT_FOUND', async () => {
it('maps remote-branch-not-found to REF_NOT_FOUND', async () => {
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
'fatal: Remote branch nonexistent not found in upstream origin',
false,
));
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'BRANCH_NOT_FOUND' });
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'REF_NOT_FOUND' });
});
it('upgrades REF_NOT_FOUND to REF_DELETED when the source has prior history', async () => {
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
'fatal: Remote branch nonexistent not found in upstream origin',
false,
));
await expect(svc().fetchFromGit({ ...fetchParams, hasPriorHistory: true }))
.rejects.toMatchObject({ code: 'REF_DELETED' });
});
it('returns REF_DELETED when a resolved ref changes namespace', async () => {
mockResolveRef.mockResolvedValueOnce({ commitSha: 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb', kind: 'tag' });
await expect(svc().fetchFromGit({
...fetchParams,
priorIdentity: { commitSha: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', kind: 'branch' },
})).rejects.toMatchObject({ code: 'REF_DELETED' });
expect(mockFetchAtCommit).not.toHaveBeenCalled();
});
it('returns REF_DELETED when a branch tip moves by force-push', async () => {
mockResolveRef.mockResolvedValueOnce({ commitSha: 'cccccccccccccccccccccccccccccccccccccccc', kind: 'branch' });
mockVerifyFastForward.mockResolvedValueOnce(false);
await expect(svc().fetchFromGit({
...fetchParams,
priorIdentity: { commitSha: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', kind: 'branch' },
})).rejects.toMatchObject({ code: 'REF_DELETED' });
expect(mockFetchAtCommit).not.toHaveBeenCalled();
});
it('propagates verifyFastForward transport failures without upgrading to REF_DELETED', async () => {
mockResolveRef.mockResolvedValueOnce({ commitSha: 'cccccccccccccccccccccccccccccccccccccccc', kind: 'branch' });
mockVerifyFastForward.mockRejectedValueOnce({
transportFailure: true as const,
reason: 'timeout',
host: 'github.com',
hasToken: true,
});
await expect(svc().fetchFromGit({
...fetchParams,
priorIdentity: { commitSha: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', kind: 'branch' },
})).rejects.toMatchObject({ code: 'NETWORK_TIMEOUT' });
expect(mockFetchAtCommit).not.toHaveBeenCalled();
});
it('maps a host refusal to serve a pinned SHA to UNSUPPORTED_REF', async () => {
// A SHA fetch requires the host to serve unadvertised objects; a
// refusal (allowAnySHA1InWant off) is a server-capability failure,
// not a missing commit. GitLab/Gitea word it differently from GitHub,
// so the classifier matches the stable "unadvertised object" phrase.
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
'fatal: upload-pack: unable to find 0123456789abcdef0123456789abcdef01234567, does not allow request for unadvertised object',
false,
));
await expect(svc().fetchFromGit({ ...fetchParams, branch: '0123456789abcdef0123456789abcdef01234567' }))
.rejects.toMatchObject({ code: 'UNSUPPORTED_REF' });
});
it('keeps NETWORK_TIMEOUT on a timed-out fetch even with prior history', async () => {
// The REF_DELETED upgrade fires only on a classified REF_NOT_FOUND.
// A network timeout on a source that previously resolved must stay a
// timeout, not read as "the ref vanished".
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
"fatal: unable to access 'https://github.com/example/repo.git/': Failed to connect to github.com port 443: Connection timed out",
false,
));
await expect(svc().fetchFromGit({ ...fetchParams, hasPriorHistory: true }))
.rejects.toMatchObject({ code: 'NETWORK_TIMEOUT' });
});
it('maps GitHub not-our-ref SHA refusal to UNSUPPORTED_REF', async () => {
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
'fatal: remote error: upload-pack: not our ref 0123456789abcdef0123456789abcdef01234567',
false,
));
await expect(svc().fetchFromGit({ ...fetchParams, branch: '0123456789abcdef0123456789abcdef01234567' }))
.rejects.toMatchObject({ code: 'UNSUPPORTED_REF' });
});
it('leaves a missing pinned SHA as GIT_ERROR, not UNSUPPORTED_REF', async () => {
// A SHA the host simply has never seen surfaces as "couldn't find
// remote ref". That is a missing object, not a server-capability
// refusal, and it is deliberately not collapsed into the delete/force
// upgrade: there is no evidence the ref ever existed.
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
'fatal: couldn\'t find remote ref 0123456789abcdef0123456789abcdef01234567',
false,
));
await expect(svc().fetchFromGit({ ...fetchParams, branch: '0123456789abcdef0123456789abcdef01234567' }))
.rejects.toMatchObject({ code: 'GIT_ERROR' });
});
it('maps connection timeouts to NETWORK_TIMEOUT', async () => {
@@ -2965,6 +3060,7 @@ function seedDirectCandidate(stackName: string): { appId: string; generationId:
commitSha: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa',
identity,
configuredRef: 'main',
resolvedRefKind: 'branch',
candidateRelPath: 'generations/cand',
appliedRelPath: 'applied/1',
manifestVersion: 1,
@@ -25,7 +25,7 @@ import os from 'os';
import path from 'path';
import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest';
import { classifyGitFailure, isTransportFailure } from '../services/git/errors';
import { nativeGitTransport } from '../services/git/nativeGitTransport';
import { nativeGitTransport, verifyFastForward } from '../services/git/nativeGitTransport';
function gitAvailable(): boolean {
return spawnSync('git', ['--version'], { stdio: 'ignore' }).status === 0;
@@ -35,31 +35,74 @@ const FIXTURES_DIR = path.resolve(__dirname, '..', '..', '..', 'e2e', 'fixtures'
const VALID_TOKEN = 'sencho-integration-test-token-do-not-leak';
const FILE_CONTENT = 'hello from the authenticated fixture repo\n';
/**
* Build a bare repo with one committed file. Mirrors e2e/gitServer.helper.ts's
* fixture builder. Returns both the served bare dir and every scratch
* directory created along the way, so the caller can remove them all.
*/
function buildBareFixtureRepo(): { bareDir: string; scratchDirs: string[] } {
interface RichFixtureRepo {
bare: { bareDir: string; scratchDirs: string[] };
mainSha: string;
annotatedTagSha: string;
lightweightTagSha: string;
pinnedSha: string;
chainTipSha: string;
rewrittenSha: string;
}
function buildRichFixtureRepo(): RichFixtureRepo {
const srcDir = mkdtempSync(path.join(os.tmpdir(), 'sencho-git-auth-src-'));
writeFileSync(path.join(srcDir, 'hello.txt'), FILE_CONTENT);
const run = (args: string[]) => {
const r = spawnSync('git', args, { cwd: srcDir, encoding: 'utf8' });
if (r.status !== 0) throw new Error(`git ${args[0]} failed: ${r.stderr}`);
return r.stdout.trim();
};
run(['init', '-b', 'main']);
run(['config', 'user.email', 'integration-test@sencho.test']);
run(['config', 'user.name', 'Sencho Integration Test']);
run(['add', '-A']);
// Explicitly off: a developer machine or CI runner with commit.gpgsign=true
// in its global gitconfig would otherwise fail this fixture commit.
run(['-c', 'commit.gpgsign=false', 'commit', '-m', 'fixture']);
const mainSha = run(['rev-parse', 'HEAD']);
run(['tag', '-a', 'v-annotated', '-m', 'annotated release']);
const annotatedTagSha = run(['rev-parse', 'v-annotated^{commit}']);
run(['tag', 'v-light']);
const lightweightTagSha = run(['rev-parse', 'v-light']);
writeFileSync(path.join(srcDir, 'second.txt'), 'second fixture file\n');
run(['add', 'second.txt']);
run(['-c', 'commit.gpgsign=false', 'commit', '-m', 'second']);
const pinnedSha = run(['rev-parse', 'HEAD']);
run(['checkout', 'main']);
for (let i = 3; i <= 5; i += 1) {
writeFileSync(path.join(srcDir, `chain-${i}.txt`), `chain file ${i}\n`);
run(['add', `chain-${i}.txt`]);
run(['-c', 'commit.gpgsign=false', 'commit', '-m', `chain-${i}`]);
}
const chainTipSha = run(['rev-parse', 'HEAD']);
run(['checkout', '--orphan', 'rewritten']);
writeFileSync(path.join(srcDir, 'rewritten.txt'), 'rewritten history\n');
run(['add', 'rewritten.txt']);
run(['-c', 'commit.gpgsign=false', 'commit', '-m', 'rewritten']);
const rewrittenSha = run(['rev-parse', 'HEAD']);
const bareRoot = mkdtempSync(path.join(os.tmpdir(), 'sencho-git-auth-bare-'));
const bareDir = path.join(bareRoot, 'repo.git');
const clone = spawnSync('git', ['clone', '--bare', '--quiet', srcDir, bareDir], { encoding: 'utf8' });
if (clone.status !== 0) throw new Error(`git clone --bare failed: ${clone.stderr}`);
return { bareDir, scratchDirs: [srcDir, bareRoot] };
const pushRewritten = spawnSync(
'git',
['push', bareDir, 'rewritten:refs/heads/rewritten'],
{ cwd: srcDir, encoding: 'utf8' },
);
if (pushRewritten.status !== 0) {
throw new Error(`git push rewritten main failed: ${pushRewritten.stderr}`);
}
return {
bare: { bareDir, scratchDirs: [srcDir, bareRoot] },
mainSha,
annotatedTagSha,
lightweightTagSha,
pinnedSha,
chainTipSha,
rewrittenSha,
};
}
/** Serve one bare repo over HTTPS smart-HTTP, rejecting any request without a valid Basic Auth token. */
@@ -136,12 +179,13 @@ describe.skipIf(!gitAvailable())('authenticated native git transport (real git,
let closeServer: () => void;
let prevExtraCaCerts: string | undefined;
let fixtureScratchDirs: string[] = [];
let fixture: RichFixtureRepo;
const workspaces: string[] = [];
beforeAll(async () => {
const { bareDir, scratchDirs } = buildBareFixtureRepo();
fixtureScratchDirs = scratchDirs;
const served = await serveAuthedRepo(bareDir);
fixture = buildRichFixtureRepo();
fixtureScratchDirs = fixture.bare.scratchDirs;
const served = await serveAuthedRepo(fixture.bare.bareDir);
repoUrl = served.url;
closeServer = served.close;
prevExtraCaCerts = process.env.NODE_EXTRA_CA_CERTS;
@@ -197,6 +241,7 @@ describe.skipIf(!gitAvailable())('authenticated native git transport (real git,
repoUrl,
ref: 'main',
token: VALID_TOKEN,
refKind: 'branch',
commitSha: resolved.commitSha,
timeoutMs: 15_000,
workspaceRoot: fetchWorkspace,
@@ -223,6 +268,7 @@ describe.skipIf(!gitAvailable())('authenticated native git transport (real git,
repoUrl,
ref: 'main',
token: VALID_TOKEN,
refKind: 'branch',
commitSha: resolved.commitSha,
timeoutMs: 15_000,
workspaceRoot: fetchWorkspace,
@@ -277,4 +323,140 @@ describe.skipIf(!gitAvailable())('authenticated native git transport (real git,
expect(serialized).not.toContain(wrongToken);
expect(serialized).not.toContain(VALID_TOKEN);
});
it('resolves and fetches an annotated tag through the peeled commit', async () => {
const workspaceRoot = await makeWorkspace();
const resolved = await nativeGitTransport.resolveRef({
repoUrl,
ref: 'v-annotated',
token: VALID_TOKEN,
timeoutMs: 15_000,
workspaceRoot,
});
expect(resolved).toMatchObject({ commitSha: fixture.annotatedTagSha, kind: 'tag' });
const fetchWorkspace = await makeWorkspace();
const fetched = await nativeGitTransport.fetchAtCommit({
repoUrl,
ref: 'v-annotated',
token: VALID_TOKEN,
refKind: 'tag',
commitSha: resolved.commitSha,
timeoutMs: 15_000,
workspaceRoot: fetchWorkspace,
maxBytes: 10 * 1024 * 1024,
});
expect(fetched.commitSha).toBe(fixture.annotatedTagSha);
expect(await fs.readFile(path.join(fetched.dir, 'hello.txt'), 'utf8')).toBe(FILE_CONTENT);
});
it('resolves and fetches a lightweight tag', async () => {
const workspaceRoot = await makeWorkspace();
const resolved = await nativeGitTransport.resolveRef({
repoUrl,
ref: 'v-light',
token: VALID_TOKEN,
timeoutMs: 15_000,
workspaceRoot,
});
expect(resolved).toMatchObject({ commitSha: fixture.lightweightTagSha, kind: 'tag' });
const fetchWorkspace = await makeWorkspace();
const fetched = await nativeGitTransport.fetchAtCommit({
repoUrl,
ref: 'v-light',
token: VALID_TOKEN,
refKind: 'tag',
commitSha: resolved.commitSha,
timeoutMs: 15_000,
workspaceRoot: fetchWorkspace,
maxBytes: 10 * 1024 * 1024,
});
expect(fetched.commitSha).toBe(fixture.lightweightTagSha);
});
it('resolves and fetches a pinned commit SHA', async () => {
const workspaceRoot = await makeWorkspace();
const resolved = await nativeGitTransport.resolveRef({
repoUrl,
ref: fixture.chainTipSha,
token: VALID_TOKEN,
timeoutMs: 15_000,
workspaceRoot,
});
expect(resolved).toMatchObject({ commitSha: fixture.chainTipSha, kind: 'sha' });
const fetchWorkspace = await makeWorkspace();
const fetched = await nativeGitTransport.fetchAtCommit({
repoUrl,
ref: fixture.chainTipSha,
token: VALID_TOKEN,
refKind: 'sha',
commitSha: resolved.commitSha,
timeoutMs: 15_000,
workspaceRoot: fetchWorkspace,
maxBytes: 10 * 1024 * 1024,
});
expect(fetched.commitSha).toBe(fixture.chainTipSha);
expect(await fs.readFile(path.join(fetched.dir, 'chain-5.txt'), 'utf8')).toBe('chain file 5\n');
});
it('treats a linear branch advance as a fast-forward', async () => {
const workspaceRoot = await makeWorkspace();
const fastForward = await verifyFastForward({
repoUrl,
ancestorSha: fixture.pinnedSha,
descendantSha: fixture.chainTipSha,
token: VALID_TOKEN,
timeoutMs: 15_000,
workspaceRoot,
maxBytes: 10 * 1024 * 1024,
});
expect(fastForward).toBe(true);
});
it('treats a multi-commit branch advance as a fast-forward', async () => {
const workspaceRoot = await makeWorkspace();
const fastForward = await verifyFastForward({
repoUrl,
ancestorSha: fixture.mainSha,
descendantSha: fixture.chainTipSha,
token: VALID_TOKEN,
timeoutMs: 15_000,
workspaceRoot,
maxBytes: 10 * 1024 * 1024,
});
expect(fastForward).toBe(true);
});
it('rejects rewritten history as non-fast-forward', async () => {
const workspaceRoot = await makeWorkspace();
const fastForward = await verifyFastForward({
repoUrl,
ancestorSha: fixture.mainSha,
descendantSha: fixture.rewrittenSha,
token: VALID_TOKEN,
timeoutMs: 15_000,
workspaceRoot,
maxBytes: 10 * 1024 * 1024,
});
expect(fastForward).toBe(false);
});
it('classifies verifyFastForward auth failures without collapsing to non-fast-forward', async () => {
const workspaceRoot = await makeWorkspace();
const failure = await verifyFastForward({
repoUrl,
ancestorSha: fixture.mainSha,
descendantSha: fixture.chainTipSha,
token: 'wrong-token',
timeoutMs: 15_000,
workspaceRoot,
maxBytes: 10 * 1024 * 1024,
}).then(() => null, (e: unknown) => e);
expect(isTransportFailure(failure)).toBe(true);
if (!isTransportFailure(failure)) throw new Error('unreachable');
expect(classifyGitFailure(failure).code).toBe('AUTH_FAILED');
});
});
+417 -3
View File
@@ -9,7 +9,7 @@
* flow, and the size watchdog.
*/
import { EventEmitter } from 'events';
import { promises as fs, rmSync } from 'fs';
import { promises as fs, rmSync, existsSync } from 'fs';
import os from 'os';
import path from 'path';
import { beforeEach, describe, expect, it, vi } from 'vitest';
@@ -39,7 +39,7 @@ import {
writeCredentialHelper,
} from '../services/git/credentialHelper';
import * as gitBinary from '../services/git/gitBinary';
import { nativeGitTransport, REF_MAX_LEN, startSizeWatchdog } from '../services/git/nativeGitTransport';
import { nativeGitTransport, REF_MAX_LEN, startSizeWatchdog, verifyFastForward } from '../services/git/nativeGitTransport';
const GIT_EXEC_PATH_STUB = 'C:/Program Files/Git/mingw64/libexec/git-core';
@@ -185,12 +185,37 @@ describe('classifyGitFailure (native git stderr corpus)', () => {
it.each([
['size', { transportFailure: true as const, reason: 'size', maxBytes: 5 * 1024 * 1024, host: 'h', hasToken: false }, 'Repository exceeds the maximum clone size of 5 MB.'],
['tip-changed', { transportFailure: true as const, reason: 'tip-changed', host: 'h', hasToken: false }, 'Repository tip changed during fetch; retry the pull.'],
['ref-not-found', { transportFailure: true as const, reason: 'ref-not-found', host: 'h', hasToken: false }, 'Branch not found in the repository.'],
['ref-not-found', { transportFailure: true as const, reason: 'ref-not-found', host: 'h', hasToken: false }, 'The configured branch, tag, or commit was not found in the repository.'],
['unsupported-ref', { transportFailure: true as const, reason: 'unsupported-ref', host: 'h', hasToken: false }, 'The configured commit is not reachable on this repository host. Use a branch or tag, or a commit the host advertises.'],
['timeout', { transportFailure: true as const, reason: 'timeout', host: 'github.com', hasToken: false }, 'Timed out reaching github.com.'],
] as const)('maps structured reason %s verbatim', (_label, failure, message) => {
expect(classifyGitFailure(failure)).toMatchObject({ message });
});
it('classifies a server SHA-fetch refusal as UNSUPPORTED_REF', () => {
const c = classifyGitFailure({
transportFailure: true as const,
reason: 'exit',
stderr: "error: Server does not allow request for unadvertised object 3b18e5d",
exitCode: 128,
host: 'github.com',
hasToken: false,
});
expect(c.code).toBe('UNSUPPORTED_REF');
});
it('classifies GitHub not-our-ref SHA refusal as UNSUPPORTED_REF', () => {
const c = classifyGitFailure({
transportFailure: true as const,
reason: 'exit',
stderr: 'fatal: remote error: upload-pack: not our ref abcdef0123456789abcdef0123456789abcdef',
exitCode: 128,
host: 'github.com',
hasToken: true,
});
expect(c.code).toBe('UNSUPPORTED_REF');
});
it('scrubs credentials from the generic fallback tail', () => {
const c = classifyGitFailure({
transportFailure: true as const,
@@ -570,6 +595,7 @@ describe('resolve/fetch/verify flow', () => {
const result = await nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
@@ -601,6 +627,7 @@ describe('resolve/fetch/verify flow', () => {
await expect(nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
@@ -611,6 +638,64 @@ describe('resolve/fetch/verify flow', () => {
}
});
it('fetches a tag through a bare --branch name', async () => {
scriptSpawn([
{ code: 0 }, // clone
{ stdout: `${SHA_A}\n` }, // rev-parse HEAD
]);
const root = await makeWorkspace();
try {
const result = await nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'v1',
refKind: 'tag',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
});
expect(result.commitSha).toBe(SHA_A);
const cloneArgs = spawnArgs(0);
const branchIdx = cloneArgs.indexOf('--branch');
expect(branchIdx).toBeGreaterThan(-1);
expect(cloneArgs[branchIdx + 1]).toBe('v1');
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('fetches a pinned SHA via init + fetch + detached checkout', async () => {
scriptSpawn([
{ code: 0 }, // init
{ code: 0 }, // fetch <sha>
{ code: 0 }, // checkout --detach
{ stdout: `${SHA_A}\n` }, // rev-parse HEAD
]);
const root = await makeWorkspace();
try {
const result = await nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: SHA_A,
refKind: 'sha',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
});
expect(result.commitSha).toBe(SHA_A);
const initArgs = spawnArgs(0);
expect(initArgs).toContain('init');
const fetchArgs = spawnArgs(1);
expect(fetchArgs).toContain('fetch');
expect(fetchArgs).toContain(SHA_A);
const checkoutArgs = spawnArgs(2);
expect(checkoutArgs).toContain('checkout');
expect(checkoutArgs).toContain('--detach');
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('raises ref-not-found when ls-remote lists no matching head', async () => {
scriptSpawn([{ stdout: '' }]);
const root = await makeWorkspace();
@@ -627,6 +712,73 @@ describe('resolve/fetch/verify flow', () => {
}
});
it('resolves a branch over a tag of the same name and records kind branch', async () => {
scriptSpawn([{
stdout: `${SHA_A}\trefs/heads/release\n${SHA_B}\trefs/tags/release\n`,
}]);
const root = await makeWorkspace();
try {
await expect(nativeGitTransport.resolveRef({
repoUrl: 'https://github.com/example/repo.git',
ref: 'release',
timeoutMs: 5000,
workspaceRoot: root,
})).resolves.toMatchObject({ commitSha: SHA_A, kind: 'branch' });
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('resolves an annotated tag through the peeled ^{} commit', async () => {
scriptSpawn([{
stdout: `${SHA_B}\trefs/tags/v1\n${SHA_A}\trefs/tags/v1^{}\n`,
}]);
const root = await makeWorkspace();
try {
await expect(nativeGitTransport.resolveRef({
repoUrl: 'https://github.com/example/repo.git',
ref: 'v1',
timeoutMs: 5000,
workspaceRoot: root,
})).resolves.toMatchObject({ commitSha: SHA_A, kind: 'tag' });
const lsRemoteArgs = mockSpawn.mock.calls[0][1] as string[];
expect(lsRemoteArgs).toContain('refs/tags/v1^{}');
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('resolves a lightweight tag from its raw ref line', async () => {
scriptSpawn([{ stdout: `${SHA_A}\trefs/tags/v1\n` }]);
const root = await makeWorkspace();
try {
await expect(nativeGitTransport.resolveRef({
repoUrl: 'https://github.com/example/repo.git',
ref: 'v1',
timeoutMs: 5000,
workspaceRoot: root,
})).resolves.toMatchObject({ commitSha: SHA_A, kind: 'tag' });
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('self-resolves a full SHA without a network round trip', async () => {
const root = await makeWorkspace();
try {
await expect(nativeGitTransport.resolveRef({
repoUrl: 'https://github.com/example/repo.git',
ref: SHA_A.toUpperCase(),
timeoutMs: 5000,
workspaceRoot: root,
})).resolves.toMatchObject({ commitSha: SHA_A, kind: 'sha' });
// The SHA needs no ls-remote: the identity IS the value.
expect(mockSpawn).not.toHaveBeenCalled();
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
// Real clocks, short budget: a hanging child settles only through the
// timeout path. (Fake timers here would freeze every later real-timer
// test in this file if this test ever failed mid-way.)
@@ -733,6 +885,7 @@ describe('clone failure classification and final size gate', () => {
await expect(nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
@@ -759,6 +912,7 @@ describe('clone failure classification and final size gate', () => {
await expect(nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
@@ -789,6 +943,7 @@ describe('clone failure classification and final size gate', () => {
await expect(nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
@@ -803,6 +958,261 @@ describe('clone failure classification and final size gate', () => {
}
});
it('enforces the size cap during fast-forward verification', async () => {
scriptSpawn([
{ code: 0 },
{ code: 0 },
{ stdout: '1\n' },
{ code: 1 },
{ stdout: 'true\n' },
{ code: 0 },
{ stdout: '2\n' },
{ code: 0 },
{ code: 0 },
]);
const root = await makeWorkspace();
await fs.writeFile(path.join(root, 'blob.bin'), 'x'.repeat(64));
try {
await expect(verifyFastForward({
repoUrl: 'https://github.com/example/repo.git',
ancestorSha: SHA_B,
descendantSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 8,
})).rejects.toMatchObject({
transportFailure: true as const,
reason: 'size',
maxBytes: 8,
});
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('deepens history exponentially with a bounded number of remote fetches', async () => {
scriptSpawn([
{ code: 0 },
{ code: 0 },
{ stdout: '1\n' },
{ code: 1 },
{ stdout: 'true\n' },
{ code: 0 },
{ stdout: '2\n' },
{ code: 1 },
{ stdout: 'true\n' },
{ code: 0 },
{ stdout: '4\n' },
{ code: 0 },
{ code: 0 },
]);
const root = await makeWorkspace();
try {
await expect(verifyFastForward({
repoUrl: 'https://github.com/example/repo.git',
ancestorSha: SHA_B,
descendantSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
})).resolves.toBe(true);
const deepenArgs = mockSpawn.mock.calls
.map((call) => call[1] as string[])
.filter((args) => args.includes('fetch'));
expect(deepenArgs).toHaveLength(3);
expect(deepenArgs[1]).toContain('--deepen=1');
expect(deepenArgs[2]).toContain('--deepen=2');
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('throws a timeout when fetch rounds are exhausted before ancestry is proven', async () => {
const scripted: ScriptedOutput[] = [{ code: 0 }, { code: 0 }, { stdout: '1\n' }];
for (let i = 0; i < 13; i += 1) {
scripted.push(
{ code: 1 },
{ stdout: 'true\n' },
{ code: 0 },
{ stdout: `${i + 2}\n` },
);
}
scripted.push({ code: 1 }, { stdout: 'true\n' });
scriptSpawn(scripted);
const root = await makeWorkspace();
try {
await expect(verifyFastForward({
repoUrl: 'https://github.com/example/repo.git',
ancestorSha: SHA_B,
descendantSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
})).rejects.toMatchObject({
transportFailure: true as const,
reason: 'timeout',
});
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('classifies a hanging ancestry probe as a timeout transport failure', async () => {
mockSpawn.mockImplementation((_cmd, args) => {
const argv = args as string[];
const child = fakeChild();
if (argv.includes('cat-file')) {
return child;
}
queueMicrotask(() => {
if (argv.includes('rev-list')) {
child.stdout.emit('data', Buffer.from('1\n'));
}
if (argv.includes('rev-parse') && argv.includes('--is-shallow-repository')) {
child.stdout.emit('data', Buffer.from('true\n'));
}
child.emit('close', 0);
});
return child;
});
const root = await makeWorkspace();
try {
await expect(verifyFastForward({
repoUrl: 'https://github.com/example/repo.git',
ancestorSha: SHA_B,
descendantSha: SHA_A,
timeoutMs: 250,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
})).rejects.toMatchObject({
transportFailure: true as const,
reason: 'timeout',
});
} finally {
mockSpawn.mockReset();
await fs.rm(root, { recursive: true, force: true });
}
});
it('treats an invalid shallow-repository probe as a transport failure', async () => {
scriptSpawn([
{ code: 0 },
{ code: 0 },
{ stdout: '1\n' },
{ code: 1 },
{ stdout: 'maybe\n' },
]);
const root = await makeWorkspace();
try {
await expect(verifyFastForward({
repoUrl: 'https://github.com/example/repo.git',
ancestorSha: SHA_B,
descendantSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
})).rejects.toMatchObject({
transportFailure: true as const,
reason: 'exit',
});
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('treats merge-base operational failures as transport failures', async () => {
scriptSpawn([
{ code: 0 },
{ code: 0 },
{ stdout: '1\n' },
{ code: 0 },
{ code: 128, stderr: 'fatal: bad object\n' },
]);
const root = await makeWorkspace();
try {
await expect(verifyFastForward({
repoUrl: 'https://github.com/example/repo.git',
ancestorSha: SHA_B,
descendantSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
})).rejects.toMatchObject({
transportFailure: true as const,
reason: 'exit',
exitCode: 128,
});
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('throws a timeout when deepen stagnates while history remains shallow', async () => {
scriptSpawn([
{ code: 0 },
{ code: 0 },
{ stdout: '1\n' },
{ code: 1 },
{ stdout: 'true\n' },
{ code: 0 },
{ stdout: '1\n' },
{ stdout: 'true\n' },
]);
const root = await makeWorkspace();
try {
await expect(verifyFastForward({
repoUrl: 'https://github.com/example/repo.git',
ancestorSha: SHA_B,
descendantSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
})).rejects.toMatchObject({
transportFailure: true as const,
reason: 'timeout',
});
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('removes the fast-forward scratch repo so shared workspace size checks stay accurate', async () => {
scriptSpawn([
{ code: 0 },
{ code: 0 },
{ stdout: '1\n' },
{ code: 1 },
{ stdout: 'true\n' },
{ code: 0 },
{ stdout: '2\n' },
{ code: 0 },
{ code: 0 },
]);
const root = await makeWorkspace();
try {
await expect(verifyFastForward({
repoUrl: 'https://github.com/example/repo.git',
ancestorSha: SHA_B,
descendantSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
maxBytes: 100 * 1024 * 1024,
})).resolves.toBe(true);
expect(existsSync(path.join(root, 'ff-check'))).toBe(false);
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('reports a clone-phase timeout and kills the child', async () => {
scriptSpawnHanging();
const root = await makeWorkspace();
@@ -811,6 +1221,7 @@ describe('clone failure classification and final size gate', () => {
await expect(nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 250,
workspaceRoot: root,
@@ -849,6 +1260,7 @@ describe('clone failure classification and final size gate', () => {
await expect(nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
@@ -887,6 +1299,7 @@ describe('clone failure classification and final size gate', () => {
await expect(nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 5000,
workspaceRoot: root,
@@ -1087,6 +1500,7 @@ describe('clone failure classification and final size gate', () => {
await expect(nativeGitTransport.fetchAtCommit({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
refKind: 'branch',
commitSha: SHA_A,
timeoutMs: 30_000,
workspaceRoot: root,
@@ -286,6 +286,7 @@ function directApp(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -333,6 +334,7 @@ function generation(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: id,
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 0,
@@ -586,6 +586,7 @@ function inlineApp(id: string, blueprintId: number): GitOpsApplicationRow {
materialization_fingerprint: null,
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -376,6 +376,7 @@ function creatingApp(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -423,6 +424,7 @@ function gen(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: SHA,
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 1,
@@ -61,6 +61,7 @@ describe('gitops create-from-git', () => {
expect(app.lifecycle_status).toBe('creating');
expect(app.desired_commit_sha).toBe(SHA);
expect(app.fetched_commit_sha).toBe(SHA);
expect(app.fetched_resolved_ref_kind).toBe('branch');
expect(app.candidate_generation_id).toBe('gen-create');
expect(app.accepted_generation_id).toBeNull();
expect(app.source_acceptance_ref).toBeNull();
@@ -679,6 +680,7 @@ function creatingApp(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -726,6 +728,7 @@ function gen(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: SHA,
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 0,
@@ -292,6 +292,7 @@ function app(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -339,6 +340,7 @@ function gen(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: 'abc123',
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 0,
@@ -940,6 +940,7 @@ function app(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -992,6 +993,7 @@ function gen(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: 'abc123',
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 0,
@@ -21,9 +21,10 @@ import path from 'path';
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
const { mockResolveRef, mockFetchAtCommit, mockGitClone, mockGitLog, compose } = vi.hoisted(() => ({
const { mockResolveRef, mockFetchAtCommit, mockVerifyFastForward, mockGitClone, mockGitLog, compose } = vi.hoisted(() => ({
mockResolveRef: vi.fn(),
mockFetchAtCommit: vi.fn(),
mockVerifyFastForward: vi.fn(),
mockGitClone: vi.fn(),
mockGitLog: vi.fn(),
/** Exit code the next daemon-dependent compose command reports. */
@@ -39,6 +40,7 @@ vi.mock('../services/git/nativeGitTransport', () => ({
resolveRef: mockResolveRef,
fetchAtCommit: mockFetchAtCommit,
},
verifyFastForward: mockVerifyFastForward,
}));
/**
@@ -147,10 +149,11 @@ let projectApplication: typeof import('../services/gitops/derive').projectApplic
* wireTransportDefaults.
*/
function wireTransportDefaults(): void {
mockVerifyFastForward.mockResolvedValue(true);
mockResolveRef.mockImplementation(async () => {
const log = await mockGitLog({});
const oid = Array.isArray(log) ? log[0]?.oid : undefined;
return { commitSha: oid ?? '' };
return { commitSha: oid ?? '', kind: 'branch' as const };
});
mockFetchAtCommit.mockImplementation(async (req: { workspaceRoot: string; commitSha: string }) => {
const dir = path.join(req.workspaceRoot, 'repo');
@@ -202,6 +205,7 @@ describe('Direct Git producers drive the revision state', () => {
beforeEach(() => {
mockResolveRef.mockReset();
mockFetchAtCommit.mockReset();
mockVerifyFastForward.mockReset();
mockGitClone.mockReset();
mockGitLog.mockReset();
wireTransportDefaults();
@@ -508,6 +508,7 @@ function application(): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -140,6 +140,7 @@ function creatingApp(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -124,6 +124,7 @@ function app(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -171,6 +172,7 @@ function gen(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: id,
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 0,
@@ -438,6 +438,7 @@ function app(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -485,6 +486,7 @@ function gen(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: 'abc123',
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 0,
@@ -0,0 +1,65 @@
/**
* Legacy installations regain gitops_generations.resolved_ref_kind and
* gitops_applications.fetched_resolved_ref_kind through initSchema's maybeAddCol.
*/
import fs from 'fs';
import os from 'os';
import path from 'path';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { BASELINE_DB_PATH } from './helpers/testConstants';
import { DatabaseService } from '../services/DatabaseService';
function resetDatabaseSingleton(): void {
const holder = DatabaseService as unknown as { instance?: DatabaseService };
const existing = holder.instance;
if (existing) {
try {
existing.getDb().close();
} catch {
// already closed
}
holder.instance = undefined;
}
}
let tmpDir: string;
beforeAll(async () => {
vi.resetModules();
tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'sencho-gitops-ref-mig-'));
process.env.DATA_DIR = tmpDir;
const composeDir = path.join(tmpDir, 'compose');
fs.mkdirSync(composeDir, { recursive: true });
process.env.COMPOSE_DIR = composeDir;
fs.copyFileSync(BASELINE_DB_PATH, path.join(tmpDir, 'sencho.db'));
const Database = (await import('better-sqlite3')).default;
const raw = new Database(path.join(tmpDir, 'sencho.db'));
raw.exec('ALTER TABLE gitops_generations DROP COLUMN resolved_ref_kind');
raw.exec('ALTER TABLE gitops_applications DROP COLUMN fetched_resolved_ref_kind');
raw.close();
const { DatabaseService } = await import('../services/DatabaseService');
resetDatabaseSingleton();
DatabaseService.getInstance();
});
afterAll(() => {
resetDatabaseSingleton();
fs.rmSync(tmpDir, { recursive: true, force: true });
});
describe('resolved ref kind schema migration', () => {
it('re-adds generation and application columns through DatabaseService initSchema', async () => {
const { DatabaseService } = await import('../services/DatabaseService');
const db = DatabaseService.getInstance().getDb();
const generationCols = new Set(
(db.pragma('table_info(gitops_generations)') as Array<{ name: string }>).map((c) => c.name),
);
const applicationCols = new Set(
(db.pragma('table_info(gitops_applications)') as Array<{ name: string }>).map((c) => c.name),
);
expect(generationCols.has('resolved_ref_kind')).toBe(true);
expect(applicationCols.has('fetched_resolved_ref_kind')).toBe(true);
});
});
@@ -221,6 +221,36 @@ describe('gitops schema', () => {
});
expect(store.getArtifactSet('art-1')?.qualification).toBe('unresolved');
});
it('round-trips resolved_ref_kind for a tag-resolved generation', async () => {
const store = GitOpsStore.getInstance();
store.insertApplication(directApp('app-tag', 'tag-web'));
const { DatabaseService } = await import('../services/DatabaseService');
const raw = DatabaseService.getInstance().getDb();
raw.prepare("UPDATE gitops_applications SET configured_ref = 'v1' WHERE id = 'app-tag'").run();
store.insertGeneration({
...generation('gen-tag', 'app-tag'),
commit_sha: 'abc123',
configured_ref: 'v1',
resolved_ref_kind: 'tag',
});
const row = store.getGeneration('gen-tag');
expect(row?.resolved_ref_kind).toBe('tag');
expect(row?.configured_ref).toBe('v1');
});
it('round-trips fetched_resolved_ref_kind on application fetch transitions', async () => {
const store = GitOpsStore.getInstance();
store.insertApplication(directApp('app-fetch-kind', 'fetch-web'));
const { GitOpsTransitions } = await import('../services/gitops/transitions');
const tx = GitOpsTransitions.getInstance();
const env = { operationId: 'op-fetch-kind', actor: 'tester', trigger: 'manual', at: Date.now() };
tx.fetchStarted('app-fetch-kind', env);
tx.fetched('app-fetch-kind', 'abc123', env, 'tag');
const app = store.getApplication('app-fetch-kind');
expect(app?.fetched_commit_sha).toBe('abc123');
expect(app?.fetched_resolved_ref_kind).toBe('tag');
});
});
function directApp(id: string, stackName: string): GitOpsApplicationRow {
@@ -241,6 +271,7 @@ function directApp(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -314,6 +345,7 @@ function generation(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: 'abc123',
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 0,
@@ -574,6 +574,7 @@ function app(id: string, stackName: string): GitOpsApplicationRow {
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -621,6 +622,7 @@ function gen(id: string, applicationId: string): GitOpsGenerationRow {
application_id: applicationId,
commit_sha: 'abc123',
repo_url: 'https://github.com/org/repo.git',
resolved_ref_kind: 'branch',
configured_ref: 'main',
repo_identity_json: '{"host":"github.com","pathname":"/org/repo.git"}',
manifest_version: 0,
@@ -34,6 +34,7 @@ export function directApplicationFixture(id: string, stackName: string): GitOpsA
materialization_fingerprint: 'a'.repeat(64),
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
+4 -4
View File
@@ -39,7 +39,7 @@ async function handleBrowse(req: Request, res: Response, storedToken: string | n
return;
}
if (typeof branch !== 'string' || !branch.trim()) {
res.status(400).json({ error: 'branch is required' });
res.status(400).json({ error: 'A branch, tag, or commit SHA is required.' });
return;
}
const repoUrlError = repoUrlRejectionMessage(repo_url);
@@ -48,7 +48,7 @@ async function handleBrowse(req: Request, res: Response, storedToken: string | n
return;
}
if (branch.length > MAX_BRANCH_LENGTH) {
res.status(400).json({ error: 'branch is too long' });
res.status(400).json({ error: 'The branch, tag, or commit SHA is too long.' });
return;
}
if (auth_type !== undefined && auth_type !== 'none' && auth_type !== 'token') {
@@ -229,7 +229,7 @@ stackGitSourceRouter.put('/:stackName/git-source', async (req: Request, res: Res
return;
}
if (typeof branch !== 'string' || !branch.trim()) {
res.status(400).json({ error: 'branch is required' });
res.status(400).json({ error: 'A branch, tag, or commit SHA is required.' });
return;
}
const selection = parseComposeSelection(req.body);
@@ -255,7 +255,7 @@ stackGitSourceRouter.put('/:stackName/git-source', async (req: Request, res: Res
return;
}
if (branch.length > MAX_BRANCH_LENGTH) {
res.status(400).json({ error: 'branch is too long' });
res.status(400).json({ error: 'The branch, tag, or commit SHA is too long.' });
return;
}
if (typeof env_path === 'string' && env_path.length > MAX_ENV_PATH_LENGTH) {
+4
View File
@@ -1950,6 +1950,10 @@ export class DatabaseService {
maybeAddCol('stack_update_recovery_generations', 'content_path', 'TEXT');
maybeAddCol('stack_update_recovery_generations', 'operation_kind', 'TEXT');
maybeAddCol('stack_update_cleanup_pending', 'required_blueprint_id', 'INTEGER');
// Resolved ref kind for existing GitOps generations. New installs get it
// from the CREATE TABLE; older DBs need the additive column here.
maybeAddCol('gitops_generations', 'resolved_ref_kind', 'TEXT NULL');
maybeAddCol('gitops_applications', 'fetched_resolved_ref_kind', 'TEXT NULL');
// Distributed API model columns
maybeAddCol('nodes', 'api_url', "TEXT DEFAULT ''");
+93 -9
View File
@@ -29,7 +29,8 @@ import type { GitChangePlan, PublicGitChangePlan, GitChangePlanCounts, PublicGit
import { GIT_CHANGE_PLAN_SCHEMA_VERSION } from '../types/gitChangePlan';
import type { NotificationCategory } from './NotificationService';
import { classifyGitFailure, isTransportFailure } from './git/errors';
import { nativeGitTransport } from './git/nativeGitTransport';
import type { RefKind } from './git/types';
import { nativeGitTransport, verifyFastForward } from './git/nativeGitTransport';
import { GitOpsStore } from './gitops/store';
import { GitOpsTransitions, GitOpsTransitionError } from './gitops/transitions';
import {
@@ -58,7 +59,9 @@ import { managedAreaBase } from './gitops/managedPaths';
export type GitSourceErrorCode =
| 'REPO_NOT_FOUND'
| 'AUTH_FAILED'
| 'BRANCH_NOT_FOUND'
| 'REF_NOT_FOUND'
| 'REF_DELETED'
| 'UNSUPPORTED_REF'
| 'FILE_NOT_FOUND'
| 'NETWORK_TIMEOUT'
| 'GIT_ERROR'
@@ -111,12 +114,29 @@ export interface FetchParams {
* Used by the pull/create paths for complete-project materialization.
*/
onClone?: (cloneDir: string, commitSha: string, envContent: string | null) => Promise<unknown>;
/**
* True when this source has fetched successfully before. Turns a ref that
* now fails to resolve into REF_DELETED (removed or force-pushed) instead
* of a plain REF_NOT_FOUND, which would read as a mis-typed ref.
*/
hasPriorHistory?: boolean;
/**
* The commit and resolved namespace from the last successful fetch. Used to
* detect force-pushes and ref-kind changes when the symbolic ref still exists.
*/
priorIdentity?: { commitSha: string; kind: RefKind };
}
export interface FetchResult {
composeFiles: ComposeFile[];
envContent: string | null;
commitSha: string;
/**
* The namespace the configured ref resolved through. Recorded wherever the
* commit is persisted so "tag v1 -> <sha>" and "branch v1 -> <sha>" stay
* distinguishable in revision state.
*/
resolvedRefKind: RefKind;
/**
* Non-fatal issues detected during the fetch (e.g. the repo uses
* submodules that are not cloned). The stack is still usable but the
@@ -407,6 +427,24 @@ async function readRepoFile(rootDir: string, relPath: string, label: string): Pr
const SUBMODULE_WARNING =
'Repository contains Git submodules. Their contents are not cloned; any paths referenced from them will be missing at deploy time.';
const REF_DELETED_MESSAGE =
'The configured branch, tag, or commit no longer points at the same revision as before. It may have been deleted, force-pushed, or moved to a different commit (for example a retagged release).';
function priorFetchIdentity(app: GitOpsApplicationRow | null | undefined): FetchParams['priorIdentity'] {
if (!app?.fetched_commit_sha) return undefined;
let kind = app.fetched_resolved_ref_kind;
if (!kind) {
const genId = app.candidate_generation_id ?? app.accepted_generation_id;
if (genId) {
kind = GitOpsStore.getInstance().getGeneration(genId)?.resolved_ref_kind ?? null;
}
}
return {
commitSha: app.fetched_commit_sha,
kind: kind ?? 'branch',
};
}
/**
* Reject any relative path that resolves into the `.git` metadata
* directory. The path-traversal check in `fetchFromGit` already bounds
@@ -917,12 +955,20 @@ export class GitSourceService {
* structured transport failures mapped below.
*/
private async withClonedRepo<T>(
params: { repoUrl: string; branch: string; token?: string | null; timeoutMs?: number },
fn: (dir: string, commitSha: string, warnings: string[]) => Promise<T>,
params: {
repoUrl: string;
branch: string;
token?: string | null;
timeoutMs?: number;
hasPriorHistory?: boolean;
priorIdentity?: { commitSha: string; kind: RefKind };
},
fn: (dir: string, commitSha: string, warnings: string[], resolvedRefKind: RefKind) => Promise<T>,
): Promise<T> {
const { repoUrl, branch, token } = params;
const timeoutMs = params.timeoutMs ?? DEFAULT_FETCH_TIMEOUT_MS;
const root = await createTempDir();
const hasPriorHistory = params.hasPriorHistory === true || params.priorIdentity != null;
try {
const resolved = await nativeGitTransport.resolveRef({
@@ -932,9 +978,30 @@ export class GitSourceService {
timeoutMs,
workspaceRoot: root,
});
if (params.priorIdentity) {
const prior = params.priorIdentity;
if (prior.kind !== resolved.kind) {
throw new GitSourceError('REF_DELETED', REF_DELETED_MESSAGE);
}
if (prior.kind !== 'sha' && prior.commitSha !== resolved.commitSha) {
const fastForward = await verifyFastForward({
repoUrl,
ancestorSha: prior.commitSha,
descendantSha: resolved.commitSha,
token,
timeoutMs,
workspaceRoot: root,
maxBytes: maxCloneBytes(),
});
if (!fastForward) {
throw new GitSourceError('REF_DELETED', REF_DELETED_MESSAGE);
}
}
}
const fetched = await nativeGitTransport.fetchAtCommit({
repoUrl,
ref: branch,
refKind: resolved.kind,
token,
timeoutMs,
commitSha: resolved.commitSha,
@@ -951,7 +1018,7 @@ export class GitSourceService {
warnings.push(SUBMODULE_WARNING);
}
return await fn(fetched.dir, fetched.commitSha, warnings);
return await fn(fetched.dir, fetched.commitSha, warnings, resolved.kind);
} catch (e) {
if (isTransportFailure(e)) {
// The classified message operators see is deliberately
@@ -966,6 +1033,11 @@ export class GitSourceService {
console.error(`[GitSource:transport] argv=[${e.argv.map((a) => sanitizeForLog(a)).join(' ')}]`);
}
const classified = classifyGitFailure(e);
// A ref that resolved before but no longer does is a deletion
// or force-push, distinct from a mis-typed ref on first link.
if (classified.code === 'REF_NOT_FOUND' && hasPriorHistory) {
throw new GitSourceError('REF_DELETED', REF_DELETED_MESSAGE);
}
throw new GitSourceError(classified.code, classified.message);
}
throw e;
@@ -993,7 +1065,14 @@ export class GitSourceService {
}
try {
return await this.withClonedRepo({ repoUrl, branch, token, timeoutMs: params.timeoutMs }, async (dir, commitSha, warnings) => {
return await this.withClonedRepo({
repoUrl,
branch,
token,
timeoutMs: params.timeoutMs,
hasPriorHistory: params.hasPriorHistory,
priorIdentity: params.priorIdentity,
}, async (dir, commitSha, warnings, resolvedRefKind) => {
const composeFiles: ComposeFile[] = [];
for (const composePath of composePaths) {
const content = await readRepoFile(dir, composePath, 'Compose path');
@@ -1040,7 +1119,7 @@ export class GitSourceService {
`[GitSource:diag] fetch ok host=${sanitizeForLog(repoHost(repoUrl))} branch=${sanitizeForLog(branch)} sha=${commitSha.slice(0, 7)} files=${composeFiles.length} env=${envContent !== null ? 'present' : 'absent'} warnings=${warnings.length} materialized=${materialization !== null} elapsedMs=${Date.now() - startedAt}`
);
}
return { composeFiles, envContent, commitSha, warnings, materialization };
return { composeFiles, envContent, commitSha, resolvedRefKind, warnings, materialization };
});
} catch (err) {
if (diag) {
@@ -1739,12 +1818,15 @@ export class GitSourceService {
const materialization: { value: MaterializationResult | null } = { value: null };
// Every throw from here on, including this fetch, is closed by the
// caller's handler, so nothing is recorded locally.
const priorIdentity = priorFetchIdentity(gitopsApp);
const fetched: FetchResult = await this.fetchFromGit({
repoUrl: src.repo_url,
branch: src.branch,
composePaths: src.compose_paths,
envPath: src.sync_env ? src.env_path : null,
token,
hasPriorHistory: priorIdentity != null,
priorIdentity,
onClone: async (cloneDir, commitSha, envContent) => {
materialization.value = await this.buildMaterialization(stackName, cloneDir, commitSha, src, envContent);
},
@@ -1792,10 +1874,10 @@ export class GitSourceService {
// generation while the projection reports the older commit.
DatabaseService.getInstance().getDb().transaction(() => {
if (!validation.ok) {
tx.fetchedInvalid(gitopsApp.id, fetched.commitSha, gitopsEnv);
tx.fetchedInvalid(gitopsApp.id, fetched.commitSha, gitopsEnv, fetched.resolvedRefKind);
return;
}
tx.fetched(gitopsApp.id, fetched.commitSha, gitopsEnv);
tx.fetched(gitopsApp.id, fetched.commitSha, gitopsEnv, fetched.resolvedRefKind);
if (!materialization.value) return;
const identity = directSourceIdentity({
repoUrl: src.repo_url,
@@ -1831,6 +1913,7 @@ export class GitSourceService {
commitSha: fetched.commitSha,
identity,
configuredRef: src.branch,
resolvedRefKind: fetched.resolvedRefKind,
candidateRelPath: materialization.value.candidateRelPath,
appliedRelPath: appliedRelPathFor(fetched.commitSha, nextManifestVersion),
manifestVersion: nextManifestVersion,
@@ -2725,6 +2808,7 @@ export class GitSourceService {
commitSha: fetched.commitSha,
identity: gitopsIdentity,
configuredRef: input.branch,
resolvedRefKind: fetched.resolvedRefKind,
candidateRelPath: staged.candidateRelPath,
appliedRelPath: appliedRelPathFor(fetched.commitSha, completeProjectManifest.manifestVersion),
manifestVersion: completeProjectManifest.manifestVersion,
+17 -4
View File
@@ -18,7 +18,8 @@
export type TransportFacingCode =
| 'REPO_NOT_FOUND'
| 'AUTH_FAILED'
| 'BRANCH_NOT_FOUND'
| 'REF_NOT_FOUND'
| 'UNSUPPORTED_REF'
| 'NETWORK_TIMEOUT'
| 'GIT_ERROR';
@@ -29,6 +30,7 @@ export type TransportFailureReason =
| 'git-missing'
| 'git-old'
| 'ref-not-found'
| 'unsupported-ref'
| 'tip-changed'
| 'size'
| 'timeout'
@@ -52,6 +54,7 @@ export type TransportFailure = TransportFailureBase & (
| { reason: 'git-missing'; stderr?: string }
| { reason: 'git-old'; stderr?: string }
| { reason: 'ref-not-found' }
| { reason: 'unsupported-ref' }
| { reason: 'tip-changed' }
| { reason: 'size'; maxBytes: number }
| { reason: 'timeout' }
@@ -103,13 +106,15 @@ export function classifyGitFailure(
case 'invalid-url':
return { code: 'GIT_ERROR', message: 'Unsupported repository URL. Use an https:// URL without embedded credentials.' };
case 'invalid-ref':
return { code: 'GIT_ERROR', message: 'Unsupported branch name. Use the branch name as the remote reports it.' };
return { code: 'GIT_ERROR', message: 'Unsupported ref name. Use a branch name, a tag name, or a full commit SHA as the remote reports it.' };
case 'git-missing':
return { code: 'GIT_ERROR', message: failure.stderr || 'The git command was not found on PATH.' };
case 'git-old':
return { code: 'GIT_ERROR', message: failure.stderr || 'The installed git client is too old.' };
case 'ref-not-found':
return { code: 'BRANCH_NOT_FOUND', message: 'Branch not found in the repository.' };
return { code: 'REF_NOT_FOUND', message: 'The configured branch, tag, or commit was not found in the repository.' };
case 'unsupported-ref':
return { code: 'UNSUPPORTED_REF', message: 'The configured commit is not reachable on this repository host. Use a branch or tag, or a commit the host advertises.' };
case 'tip-changed':
return { code: 'GIT_ERROR', message: 'Repository tip changed during fetch; retry the pull.' };
case 'size':
@@ -145,7 +150,15 @@ export function classifyGitFailure(
};
}
if (/remote branch .+ not found in upstream|branch not found/.test(raw)) {
return { code: 'BRANCH_NOT_FOUND', message: 'Branch not found in the repository.' };
return { code: 'REF_NOT_FOUND', message: 'The configured branch, tag, or commit was not found in the repository.' };
}
// A host that refuses to serve an unadvertised object (SHA fetch without
// allowAnySHA1InWant/allowReachableSHA1InWant) still exits non-zero, but
// the failure is about server capability, not the SHA existing. Hosts word
// the refusal differently (GitHub vs GitLab/Gitea), so match stable phrases
// rather than one vendor's full sentence.
if (/unadvertised object|not our ref/.test(raw)) {
return { code: 'UNSUPPORTED_REF', message: 'The configured commit is not reachable on this repository host. Use a branch or tag, or a commit the host advertises.' };
}
if (/repository[\s\S]*\bnot found\b|not found in upstream/.test(raw)) {
return {
+337 -56
View File
@@ -10,7 +10,7 @@ import {
writeCredentialHelper,
} from './credentialHelper';
import { isTransportFailure, type TransportFailure } from './errors';
import type { FetchRequest, FetchResult, GitTransport, ResolveRequest } from './types';
import type { FetchRequest, FetchResult, GitTransport, ResolveRequest, ResolveResult } from './types';
/**
* Native git transport: every Git operation is an `execFile`-style spawn of
@@ -590,26 +590,31 @@ async function ensureBinaryReady(hasToken: boolean): Promise<void> {
}
}
function parseLsRemoteLine(line: string, fullRef: string): string | null {
const tabIndex = line.indexOf('\t');
if (tabIndex === -1) return null;
if (line.slice(tabIndex + 1).trim() !== fullRef) return null;
const sha = line.slice(0, tabIndex).trim();
return SHA_PATTERN.test(sha) ? sha.toLowerCase() : null;
interface ResolvedRemoteRefs {
branchSha: string | null;
tagSha: string | null;
}
async function lsRemoteHead(
/**
* Ask the remote where a bare ref name lives. One `ls-remote` with explicit
* refspecs for both namespaces keeps the response tiny (a name matches at
* most a couple of lines even on huge repos), so the stdout cap can never
* truncate the answer we need. For an annotated tag the peeled `^{}` entry
* carries the commit, so it wins over the raw tag-object line; a lightweight
* tag's raw line already points at the commit.
*/
async function lsRemoteRefs(
url: URL,
ref: string,
env: NodeJS.ProcessEnv,
baseArgs: string[],
timeoutMs: number,
hasToken: boolean,
): Promise<string> {
): Promise<ResolvedRemoteRefs> {
let res: RunResult;
try {
res = await runGit(
[...baseArgs, 'ls-remote', '--heads', url.href, `refs/heads/${ref}`],
[...baseArgs, 'ls-remote', url.href, `refs/heads/${ref}`, `refs/tags/${ref}`, `refs/tags/${ref}^{}`],
{ env, timeoutMs: Math.min(timeoutMs, LS_REMOTE_MAX_MS) },
);
} catch (e) {
@@ -623,27 +628,289 @@ async function lsRemoteHead(
if (res.exitCode !== 0) {
throw { transportFailure: true as const, reason: 'exit', stderr: res.stderr, exitCode: res.exitCode, argv: baseArgs, host: url.host, hasToken } satisfies TransportFailure;
}
const fullRef = `refs/heads/${ref}`;
const found: ResolvedRemoteRefs = { branchSha: null, tagSha: null };
for (const line of res.stdout.split(/\r?\n/)) {
const sha = parseLsRemoteLine(line, fullRef);
if (sha) return sha;
const tabIndex = line.indexOf('\t');
if (tabIndex === -1) continue;
const sha = line.slice(0, tabIndex).trim();
if (!SHA_PATTERN.test(sha)) continue;
const full = line.slice(tabIndex + 1).trim();
if (full === `refs/heads/${ref}`) {
found.branchSha = sha.toLowerCase();
} else if (full === `refs/tags/${ref}^{}`) {
found.tagSha = sha.toLowerCase();
} else if (full === `refs/tags/${ref}` && found.tagSha === null) {
found.tagSha = sha.toLowerCase();
}
}
return found;
}
/** Remote fetch rounds allowed after the initial shallow tip fetch. */
const MAX_FF_FETCH_ROUNDS = 14;
/** Per-round deepen step cap for exponential backoff. */
const MAX_FF_DEEPEN_STEP = 2048;
/**
* Whether `descendantSha` is a fast-forward from `ancestorSha` on the remote.
* Distinguishes a normal branch advance from a force-push after ls-remote has
* already resolved the ref to a new tip.
*
* Fetches the descendant tip once, then deepens that shallow boundary with
* exponentially increasing steps until the prior commit is reachable, the
* downloaded history is complete, or a safety budget is exhausted. Operational
* failures throw a classified TransportFailure; only a proven non-fast-forward
* returns false.
*/
export async function verifyFastForward(req: {
repoUrl: string;
ancestorSha: string;
descendantSha: string;
token?: string | null;
timeoutMs?: number;
workspaceRoot: string;
maxBytes: number;
}): Promise<boolean> {
const ancestor = req.ancestorSha.toLowerCase();
const descendant = req.descendantSha.toLowerCase();
if (ancestor === descendant) return true;
const hasToken = Boolean(req.token);
await ensureBinaryReady(hasToken);
const url = assertValidRepoUrl(req.repoUrl, hasToken);
const timeoutMs = req.timeoutMs ?? DEFAULT_TIMEOUT_MS;
const deadline = Date.now() + timeoutMs;
const remainingMs = (): number => Math.max(1, deadline - Date.now());
const assertTimeBudget = (): void => {
if (Date.now() >= deadline) {
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
}
};
const { env, baseArgs } = await prepareInvocation(req.workspaceRoot, req.token);
const repoDir = path.join(req.workspaceRoot, 'ff-check');
await fs.mkdir(repoDir, { recursive: true });
let sizeExceeded = false;
let activeChild: ChildProcess | undefined;
let breachKill: Promise<void> | undefined;
const watchdog = startSizeWatchdog(req.workspaceRoot, req.maxBytes, () => {
sizeExceeded = true;
breachKill = killTree(activeChild);
});
const throwIfSizeExceeded = (): void => {
if (sizeExceeded) {
throw { transportFailure: true as const, reason: 'size', maxBytes: req.maxBytes, host: url.host, hasToken } satisfies TransportFailure;
}
};
try {
const materialize = async (args: string[]): Promise<RunResult> => {
assertTimeBudget();
let res: RunResult;
try {
res = await runGit(args, {
cwd: repoDir,
env,
timeoutMs: remainingMs(),
onSpawn: (child) => { activeChild = child; },
});
} catch (e) {
throwIfSizeExceeded();
if (isTimeoutError(e)) {
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
}
throw { transportFailure: true as const, reason: 'exit', stderr: e instanceof Error ? e.message : String(e), argv: args, host: url.host, hasToken } satisfies TransportFailure;
}
throwIfSizeExceeded();
if (res.exitCode !== 0) {
throw { transportFailure: true as const, reason: 'exit', stderr: res.stderr, exitCode: res.exitCode, argv: args, host: url.host, hasToken } satisfies TransportFailure;
}
return res;
};
const probeFailure = (res: RunResult, argv: string[]): never => {
throw {
transportFailure: true as const,
reason: 'exit',
stderr: res.stderr,
exitCode: res.exitCode,
argv,
host: url.host,
hasToken,
} satisfies TransportFailure;
};
const runProbe = async (args: string[]): Promise<RunResult> => {
assertTimeBudget();
try {
const res = await runGit(args, {
cwd: repoDir,
env,
timeoutMs: Math.min(remainingMs(), LS_REMOTE_MAX_MS),
});
throwIfSizeExceeded();
return res;
} catch (e) {
throwIfSizeExceeded();
if (isTimeoutError(e)) {
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
}
throw {
transportFailure: true as const,
reason: 'exit',
stderr: e instanceof Error ? e.message : String(e),
argv: args,
host: url.host,
hasToken,
} satisfies TransportFailure;
}
};
const isMissingObjectProbe = (res: RunResult): boolean => {
if (res.exitCode === 0) return false;
if (res.exitCode === 1) return true;
const err = res.stderr.toLowerCase();
return err.includes('not a valid object name')
|| err.includes('bad object')
|| err.includes('could not get');
};
await materialize([...baseArgs, 'init']);
await materialize([...baseArgs, 'fetch', '--depth=1', url.href, descendant]);
const countReachable = async (): Promise<number> => {
const argv = [...baseArgs, 'rev-list', '--count', descendant];
const listed = await runProbe(argv);
if (listed.exitCode !== 0) {
return probeFailure(listed, argv);
}
const parsed = Number.parseInt(listed.stdout.trim(), 10);
if (!Number.isFinite(parsed) || parsed < 0) {
throw {
transportFailure: true as const,
reason: 'exit',
stderr: `unexpected rev-list output: ${listed.stdout}`,
exitCode: listed.exitCode,
argv,
host: url.host,
hasToken,
} satisfies TransportFailure;
}
return parsed;
};
const isShallowRepository = async (): Promise<boolean> => {
const argv = [...baseArgs, 'rev-parse', '--is-shallow-repository'];
const shallow = await runProbe(argv);
if (shallow.exitCode !== 0) {
return probeFailure(shallow, argv);
}
const flag = shallow.stdout.trim();
if (flag === 'true') return true;
if (flag === 'false') return false;
throw {
transportFailure: true as const,
reason: 'exit',
stderr: `unexpected shallow-repository output: ${shallow.stdout}`,
exitCode: shallow.exitCode,
argv,
host: url.host,
hasToken,
} satisfies TransportFailure;
};
const isProvenAncestor = async (): Promise<boolean> => {
const argv = [...baseArgs, 'merge-base', '--is-ancestor', ancestor, descendant];
const ancestry = await runProbe(argv);
if (ancestry.exitCode === 0) return true;
if (ancestry.exitCode === 1) return false;
return probeFailure(ancestry, argv);
};
let reachableCount = await countReachable();
let fetchRounds = 1;
let deepenStep = 1;
const assertWithinSizeBudget = async (): Promise<void> => {
const finalSize = await treeSize(req.workspaceRoot).catch((e: unknown) => {
console.warn(`[GitSource:transport] final size measurement failed for ${req.workspaceRoot}, failing closed: ${e instanceof Error ? e.message : String(e)}`);
return -1;
});
if (sizeExceeded || finalSize < 0 || finalSize > req.maxBytes) {
throw { transportFailure: true as const, reason: 'size', maxBytes: req.maxBytes, host: url.host, hasToken } satisfies TransportFailure;
}
};
while (true) {
assertTimeBudget();
const ancestorArgv = [...baseArgs, 'cat-file', '-e', `${ancestor}^{commit}`];
const hasAncestor = await runProbe(ancestorArgv);
if (hasAncestor.exitCode === 0) {
await assertWithinSizeBudget();
return await isProvenAncestor();
}
if (!isMissingObjectProbe(hasAncestor)) {
return probeFailure(hasAncestor, ancestorArgv);
}
if (!(await isShallowRepository())) {
await assertWithinSizeBudget();
return false;
}
if (fetchRounds >= MAX_FF_FETCH_ROUNDS) {
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
}
const previousCount = reachableCount;
await materialize([...baseArgs, 'fetch', `--deepen=${deepenStep}`, url.href, descendant]);
fetchRounds += 1;
reachableCount = await countReachable();
if (reachableCount <= previousCount) {
if (!(await isShallowRepository())) {
await assertWithinSizeBudget();
return false;
}
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
}
deepenStep = Math.min(deepenStep * 2, MAX_FF_DEEPEN_STEP);
}
} finally {
watchdog.stop();
await awaitKillConfirmed(breachKill, `size-breach kill for ${url.host}`);
await fs.rm(repoDir, { recursive: true, force: true }).catch((e: unknown) => {
console.warn(`[GitSource:transport] failed to remove fast-forward scratch repo ${repoDir}: ${e instanceof Error ? e.message : String(e)}`);
});
}
throw { transportFailure: true as const, reason: 'ref-not-found', host: url.host, hasToken } satisfies TransportFailure;
}
export const nativeGitTransport: GitTransport = {
async resolveRef(req: ResolveRequest): Promise<{ commitSha: string }> {
async resolveRef(req: ResolveRequest): Promise<ResolveResult> {
const hasToken = Boolean(req.token);
await ensureBinaryReady(hasToken);
const url = assertValidRepoUrl(req.repoUrl, hasToken);
assertValidRef(req.ref, url.host, hasToken);
if (SHA_PATTERN.test(req.ref)) {
// A full SHA is self-resolving: the immutable identity IS the
// value, so there is nothing to look up. Reachability is verified
// at fetch, where the host either serves the object or refuses it
// (classified as UNSUPPORTED_REF).
return { commitSha: req.ref.toLowerCase(), kind: 'sha' };
}
assertValidRef(req.ref, url.host, hasToken);
const { env, baseArgs } = await prepareInvocation(req.workspaceRoot, req.token);
const commitSha = await lsRemoteHead(
const found = await lsRemoteRefs(
url, req.ref, env, baseArgs,
req.timeoutMs ?? DEFAULT_TIMEOUT_MS, hasToken,
);
return { commitSha };
if (found.branchSha) return { commitSha: found.branchSha, kind: 'branch' };
if (found.tagSha) return { commitSha: found.tagSha, kind: 'tag' };
throw { transportFailure: true as const, reason: 'ref-not-found', host: url.host, hasToken } satisfies TransportFailure;
},
async fetchAtCommit(req: FetchRequest): Promise<FetchResult> {
@@ -670,49 +937,63 @@ export const nativeGitTransport: GitTransport = {
});
try {
let cloneResult: RunResult;
try {
cloneResult = await runGit(
[
...baseArgs, 'clone',
'--depth=1', '--single-branch', '--no-tags', '--no-recurse-submodules',
'--branch', req.ref, url.href, checkout,
],
{ cwd: layout.homeDir, env, timeoutMs, onSpawn: (child) => { activeChild = child; } },
);
} catch (e) {
// A size breach wins over the timeout wording: both kills are
// ours, but the operator guidance differs.
// Run each materialization step through one failure mapper. A size
// breach wins over the timeout wording: both kills are ours, but
// the operator guidance differs. runGit resolves on any exit code,
// so a non-zero exit is classified here by its real stderr rather
// than leaking a generic GIT_ERROR upstream.
const materialize = async (args: string[]): Promise<RunResult> => {
let res: RunResult;
try {
res = await runGit(args, {
cwd: layout.homeDir, env, timeoutMs,
onSpawn: (child) => { activeChild = child; },
});
} catch (e) {
// A size breach wins over the timeout wording.
if (sizeExceeded) {
throw { transportFailure: true as const, reason: 'size', maxBytes: req.maxBytes, host: url.host, hasToken } satisfies TransportFailure;
}
if (isTimeoutError(e)) {
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
}
throw { transportFailure: true as const, reason: 'exit', stderr: e instanceof Error ? e.message : String(e), argv: args, host: url.host, hasToken } satisfies TransportFailure;
}
// A watchdog-triggered SIGKILL settles runGit's promise via the
// child's normal 'close' event (code null -> exitCode -1), not
// a rejection, so this is the common path for an in-flight
// breach and must check sizeExceeded before the generic mapping.
if (sizeExceeded) {
throw { transportFailure: true as const, reason: 'size', maxBytes: req.maxBytes, host: url.host, hasToken } satisfies TransportFailure;
}
if (isTimeoutError(e)) {
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
if (res.exitCode !== 0) {
throw { transportFailure: true as const, reason: 'exit', stderr: res.stderr, exitCode: res.exitCode, argv: args, host: url.host, hasToken } satisfies TransportFailure;
}
throw { transportFailure: true as const, reason: 'exit', stderr: e instanceof Error ? e.message : String(e), argv: [...baseArgs, 'clone'], host: url.host, hasToken } satisfies TransportFailure;
}
return res;
};
// A watchdog-triggered SIGKILL settles runGit's promise via the
// child's normal 'close' event (code null -> exitCode -1), not a
// rejection, so this branch is the common path for an in-flight
// breach and must check sizeExceeded before the generic mapping.
if (sizeExceeded) {
throw { transportFailure: true as const, reason: 'size', maxBytes: req.maxBytes, host: url.host, hasToken } satisfies TransportFailure;
}
// runGit resolves on any exit code: a failed clone (auth, missing
// repo, TLS) must classify by its real stderr here rather than
// fall through to rev-parse and surface as a generic GIT_ERROR.
if (cloneResult.exitCode !== 0) {
throw {
transportFailure: true as const,
reason: 'exit',
stderr: cloneResult.stderr,
exitCode: cloneResult.exitCode,
argv: [...baseArgs, 'clone'],
host: url.host,
hasToken,
} satisfies TransportFailure;
if (req.refKind === 'sha') {
// `--branch` cannot take a bare SHA, so a pinned commit uses a
// third strategy: init a repo, fetch exactly that object, and
// check it out detached. The host must allow fetching a direct
// SHA (GitHub does by default); a refusal surfaces as a
// non-zero `git fetch` here and classifies as UNSUPPORTED_REF.
await materialize([...baseArgs, 'init', checkout]);
await materialize([...baseArgs, '-C', checkout, 'fetch', '--depth=1', url.href, req.ref]);
await materialize([...baseArgs, '-C', checkout, 'checkout', '--detach', req.ref]);
} else {
// A bare name works for both branches and tags: `--branch`
// detaches at the named ref's commit either way, and passing a
// fully-qualified `refs/tags/<ref>` is rejected by git
// (`Remote branch ... not found`). The resolved kind is
// already pinned by ls-remote, and the rev-parse HEAD
// verification below confirms the checkout matched it.
const branchArg = req.ref;
await materialize([
...baseArgs, 'clone',
'--depth=1', '--single-branch', '--no-tags', '--no-recurse-submodules',
'--branch', branchArg, url.href, checkout,
]);
}
let actual: string;
+28 -3
View File
@@ -5,13 +5,22 @@
* configured ref to an immutable commit BEFORE any content is downloaded,
* and the fetch verifies it landed on exactly that commit. That makes
* immutable resolution structural rather than a convention callers have to
* remember. The ref field carries branch names today; widening it to tags and
* pinned SHAs later does not change either method's shape.
* remember.
*
* The configured ref is a free string: a branch name, a tag name, or a full
* commit SHA. Only a full 40/64-hex SHA is unambiguous on its own, so the
* transport resolves a bare name by asking the remote which namespace it
* lives in (branch, then tag) and returns the concrete kind it resolved
* through. That resolved kind is what callers record next to the immutable
* SHA, so "tag v1 -> <sha>" and "branch v1 -> <sha>" stay distinguishable in
* persisted revision state.
*/
export type RefKind = 'branch' | 'tag' | 'sha';
export interface ResolveRequest {
repoUrl: string;
/** Branch names today; tags and pinned SHAs may widen this later. */
/** Configured ref: a branch name, a tag name, or a full 40/64-hex commit SHA. */
ref: string;
token?: string | null;
/**
@@ -37,6 +46,14 @@ export interface FetchRequest extends ResolveRequest {
* `commitSha` pins what may be trusted.
*/
commitSha: string;
/**
* The kind resolveRef resolved `ref` through. It drives the fetch
* strategy: a branch or tag both ride `--branch <ref>` (git detaches at the
* named ref's commit either way), and a pinned SHA needs a third path
* (`git init` + `git fetch <sha>` + detached checkout), because `--branch`
* cannot take a bare SHA.
*/
refKind: RefKind;
/** Ceiling for the on-disk clone; enforced by the size watchdog. */
maxBytes: number;
}
@@ -48,7 +65,15 @@ export interface FetchResult {
}
export interface ResolveResult {
/** The immutable commit the configured ref resolved to. */
commitSha: string;
/**
* The namespace the configured ref resolved through. A bare name may be a
* branch or a tag, so this is resolved by the remote, not guessed. A full
* 40/64-hex SHA self-resolves with no network round-trip, so it always
* reports `sha`.
*/
kind: RefKind;
}
export interface GitTransport {
@@ -359,6 +359,7 @@ export function blankInlineApplication(id: string, blueprintId: number, at: numb
materialization_fingerprint: null,
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -5,6 +5,7 @@ import { MANAGED_ROOT_NAME } from './managedPaths';
import { encodeGitOpsJson } from './json';
import { materializationFingerprint } from './fingerprint';
import { parseHttpsRepoUrl, parseLegacyRepoUrl, secretFreeRepoUrl, serializeRepoIdentity, type RepoIdentity } from './repoIdentity';
import type { RefKind } from '../git/types';
import type {
GitOpsApplicationRow,
GitOpsCreateCheckpointRow,
@@ -123,6 +124,7 @@ export function buildDirectApplicationRow(args: {
materialization_fingerprint: args.identity.fingerprint,
desired_commit_sha: null,
fetched_commit_sha: null,
fetched_resolved_ref_kind: null,
candidate_generation_id: null,
accepted_generation_id: null,
candidate_plan_blocked: 0,
@@ -170,6 +172,7 @@ export function buildGenerationRow(args: {
commitSha: string;
identity: DirectSourceIdentity;
configuredRef: string;
resolvedRefKind: RefKind;
candidateRelPath: string;
appliedRelPath: string;
manifestVersion: number;
@@ -188,6 +191,7 @@ export function buildGenerationRow(args: {
commit_sha: args.commitSha,
repo_url: args.identity.repoUrl,
configured_ref: args.configuredRef,
resolved_ref_kind: args.resolvedRefKind,
repo_identity_json: encodeGitOpsJson(args.identity.identity),
manifest_version: args.manifestVersion,
candidate_dir: args.candidateRelPath,
+2
View File
@@ -195,6 +195,7 @@ function migrateAccepted(
application.desired_commit_sha = trust.commitSha;
application.fetched_commit_sha = trust.commitSha;
application.fetched_resolved_ref_kind = 'branch';
application.accepted_generation_id = generationId;
application.artifact_set_id = artifactSetId;
application.latest_artifact_set_id = artifactSetId;
@@ -207,6 +208,7 @@ function migrateAccepted(
application_id: application.id,
commit_sha: trust.commitSha,
repo_url: application.configured_repo_url ?? '',
resolved_ref_kind: 'branch',
configured_ref: source.branch,
repo_identity_json: application.repo_identity_json ?? '{}',
manifest_version: trust.manifestVersion,
+6
View File
@@ -63,6 +63,9 @@ CREATE TABLE IF NOT EXISTS gitops_applications (
materialization_fingerprint TEXT NULL,
desired_commit_sha TEXT NULL,
fetched_commit_sha TEXT NULL,
fetched_resolved_ref_kind TEXT NULL CHECK (
fetched_resolved_ref_kind IS NULL OR fetched_resolved_ref_kind IN ('branch','tag','sha')
),
candidate_generation_id TEXT NULL,
accepted_generation_id TEXT NULL,
candidate_plan_blocked INTEGER NOT NULL DEFAULT 0,
@@ -148,6 +151,9 @@ CREATE TABLE IF NOT EXISTS gitops_generations (
commit_sha TEXT NOT NULL,
repo_url TEXT NOT NULL,
configured_ref TEXT NOT NULL,
resolved_ref_kind TEXT NULL CHECK (
resolved_ref_kind IS NULL OR resolved_ref_kind IN ('branch','tag','sha')
),
repo_identity_json TEXT NOT NULL,
manifest_version INTEGER NOT NULL,
candidate_dir TEXT NOT NULL,
+6 -6
View File
@@ -356,7 +356,7 @@ export class GitOpsStore {
id, lifecycle_key, lifecycle_status, target_mode, stack_name, blueprint_id,
configured_repo_url, repo_identity_json, configured_ref, compose_paths_json,
context_dir, sync_env, env_path, materialization_fingerprint, desired_commit_sha,
fetched_commit_sha, candidate_generation_id, accepted_generation_id,
fetched_commit_sha, fetched_resolved_ref_kind, candidate_generation_id, accepted_generation_id,
candidate_plan_blocked, review_required, artifact_set_id, latest_artifact_set_id,
intent_revision_id, rollout_candidate_id, rollout_generation_id, source_acceptance_ref,
placement_approval_ref, rollout_authorization_ref, legacy_combined_approval_ref,
@@ -366,12 +366,12 @@ export class GitOpsStore {
recovery_ref, recovery_phase, interruption_stage, interruption_at,
interruption_operation_id, interruption_generation_id, evidence_fresh_at,
evidence_limitations_json, created_at, updated_at
) VALUES (${Array(54).fill('?').join(', ')})`,
) VALUES (${Array(55).fill('?').join(', ')})`,
).run(
row.id, row.lifecycle_key, row.lifecycle_status, row.target_mode, row.stack_name, row.blueprint_id,
row.configured_repo_url, row.repo_identity_json, row.configured_ref, row.compose_paths_json,
row.context_dir, row.sync_env, row.env_path, row.materialization_fingerprint, row.desired_commit_sha,
row.fetched_commit_sha, row.candidate_generation_id, row.accepted_generation_id,
row.fetched_commit_sha, row.fetched_resolved_ref_kind, row.candidate_generation_id, row.accepted_generation_id,
row.candidate_plan_blocked, row.review_required, row.artifact_set_id, row.latest_artifact_set_id,
row.intent_revision_id, row.rollout_candidate_id, row.rollout_generation_id, row.source_acceptance_ref,
row.placement_approval_ref, row.rollout_authorization_ref, row.legacy_combined_approval_ref,
@@ -387,13 +387,13 @@ export class GitOpsStore {
insertGeneration(row: GitOpsGenerationRow): void {
this.db().prepare(
`INSERT INTO gitops_generations (
id, application_id, commit_sha, repo_url, configured_ref, repo_identity_json,
id, application_id, commit_sha, repo_url, configured_ref, resolved_ref_kind, repo_identity_json,
manifest_version, candidate_dir, applied_dir, expected_invocation_json,
materialization_fingerprint, validation_ok, plan_blocked, change_plan_fingerprint,
operation_id, trigger, actor, previous_generation_id, redacted_limitations_json, created_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
).run(
row.id, row.application_id, row.commit_sha, row.repo_url, row.configured_ref, row.repo_identity_json,
row.id, row.application_id, row.commit_sha, row.repo_url, row.configured_ref, row.resolved_ref_kind, row.repo_identity_json,
row.manifest_version, row.candidate_dir, row.applied_dir, row.expected_invocation_json,
row.materialization_fingerprint, row.validation_ok, row.plan_blocked, row.change_plan_fingerprint,
row.operation_id, row.trigger, row.actor, row.previous_generation_id, row.redacted_limitations_json,
+9 -4
View File
@@ -1,3 +1,4 @@
import type { RefKind } from '../git/types';
import { DatabaseService } from '../DatabaseService';
import {
decodeArtifactEvidenceJson,
@@ -145,12 +146,13 @@ export class GitOpsTransitions {
})();
}
fetched(applicationId: string, commitSha: string, envelope: EventEnvelope): TransitionResult {
fetched(applicationId: string, commitSha: string, envelope: EventEnvelope, resolvedRefKind: RefKind | null = null): TransitionResult {
return this.mutateApp(applicationId, envelope, 'fetched', 'committed', (app) => {
this.requireMatchingFetch(app, envelope);
this.clearActive(app);
app.desired_commit_sha = commitSha;
app.fetched_commit_sha = commitSha;
app.fetched_resolved_ref_kind = resolvedRefKind;
app.retry_count = 0;
this.clearAppFailure(app, ['fetch', 'validation']);
this.clearInterruption(app, 'fetch_started');
@@ -193,12 +195,13 @@ export class GitOpsTransitions {
* from it. Retry count is deliberately not reset, because nothing about this
* outcome suggests the next attempt will differ.
*/
fetchedInvalid(applicationId: string, commitSha: string, envelope: EventEnvelope): TransitionResult {
fetchedInvalid(applicationId: string, commitSha: string, envelope: EventEnvelope, resolvedRefKind: RefKind | null = null): TransitionResult {
return this.mutateApp(applicationId, envelope, 'fetched_invalid', 'failed', (app) => {
this.requireMatchingFetch(app, envelope);
this.clearActive(app);
app.desired_commit_sha = commitSha;
app.fetched_commit_sha = commitSha;
app.fetched_resolved_ref_kind = resolvedRefKind;
app.failure_stage = 'validation';
app.failure_class = 'validation';
app.failure_at = envelope.at;
@@ -295,6 +298,7 @@ export class GitOpsTransitions {
app.materialization_fingerprint = args.material.fingerprint;
app.desired_commit_sha = null;
app.fetched_commit_sha = null;
app.fetched_resolved_ref_kind = null;
app.candidate_generation_id = null;
app.candidate_plan_blocked = 0;
app.review_required = 0;
@@ -456,6 +460,7 @@ export class GitOpsTransitions {
app.desired_commit_sha = args.commitSha;
app.fetched_commit_sha = args.commitSha;
app.fetched_resolved_ref_kind = args.generation.resolved_ref_kind;
app.retry_count = 0;
pushHistory(this.history(app, args.envelope, {
stage: 'fetched',
@@ -2235,7 +2240,7 @@ export class GitOpsTransitions {
lifecycle_status=?, configured_repo_url=?, repo_identity_json=?, configured_ref=?,
compose_paths_json=?, context_dir=?, sync_env=?, env_path=?,
materialization_fingerprint=?, desired_commit_sha=?, fetched_commit_sha=?,
candidate_generation_id=?, accepted_generation_id=?, candidate_plan_blocked=?,
fetched_resolved_ref_kind=?, candidate_generation_id=?, accepted_generation_id=?, candidate_plan_blocked=?,
review_required=?, artifact_set_id=?, latest_artifact_set_id=?,
intent_revision_id=?, rollout_candidate_id=?, rollout_generation_id=?,
source_acceptance_ref=?, placement_approval_ref=?, rollout_authorization_ref=?,
@@ -2252,7 +2257,7 @@ export class GitOpsTransitions {
app.lifecycle_status, app.configured_repo_url, app.repo_identity_json, app.configured_ref,
app.compose_paths_json, app.context_dir, app.sync_env, app.env_path,
app.materialization_fingerprint, app.desired_commit_sha, app.fetched_commit_sha,
app.candidate_generation_id, app.accepted_generation_id, app.candidate_plan_blocked,
app.fetched_resolved_ref_kind, app.candidate_generation_id, app.accepted_generation_id, app.candidate_plan_blocked,
app.review_required, app.artifact_set_id, app.latest_artifact_set_id,
app.intent_revision_id, app.rollout_candidate_id, app.rollout_generation_id,
app.source_acceptance_ref, app.placement_approval_ref, app.rollout_authorization_ref,
+4
View File
@@ -1,5 +1,6 @@
import type { ArtifactEvidenceJson, ObservedArtifactIdentity } from './json';
import type { RepoIdentity } from './repoIdentity';
import type { RefKind } from '../git/types';
export type GitOpsTargetMode = 'direct' | 'inline_blueprint' | 'blueprint';
export type GitOpsLifecycleStatus = 'active' | 'creating' | 'detached' | 'deleted';
@@ -48,6 +49,7 @@ export type GitOpsApplicationRow = {
materialization_fingerprint: string | null;
desired_commit_sha: string | null;
fetched_commit_sha: string | null;
fetched_resolved_ref_kind: RefKind | null;
candidate_generation_id: string | null;
accepted_generation_id: string | null;
candidate_plan_blocked: number;
@@ -137,6 +139,8 @@ export type GitOpsGenerationRow = {
commit_sha: string;
repo_url: string;
configured_ref: string;
/** The namespace (branch | tag | sha) the configured ref resolved through. */
resolved_ref_kind: RefKind | null;
repo_identity_json: string;
manifest_version: number;
candidate_dir: string;
+4 -1
View File
@@ -21,9 +21,12 @@ export function gitSourceStatus(code: GitSourceErrorCode): number {
case 'PLAN_FINGERPRINT_REQUIRED':
return 400;
case 'REPO_NOT_FOUND':
case 'BRANCH_NOT_FOUND':
case 'REF_NOT_FOUND':
case 'REF_DELETED':
case 'FILE_NOT_FOUND':
return 404;
case 'UNSUPPORTED_REF':
return 400;
case 'STALE_PLAN':
case 'PLAN_BLOCKED':
case 'LEGACY_PENDING':
+20 -12
View File
@@ -3,7 +3,7 @@ title: Git Sources
description: Link a stack to a Git repository and keep one or more compose files in sync via manual pulls or webhook triggers.
---
Git Sources turn any stack into a GitOps target. Point Sencho at a repository and branch, choose one or more compose files to merge in order, pull updates on demand or from CI, and review a classified change plan before applying files to disk. Optional sibling `.env` sync keeps configuration consistent too.
Git Sources turn any stack into a GitOps target. Point Sencho at a repository and ref (branch, tag, or commit SHA), choose one or more compose files to merge in order, pull updates on demand or from CI, and review a classified change plan before applying files to disk. Optional sibling `.env` sync keeps configuration consistent too.
<Note>
Git Sources are available on every tier, including Community.
@@ -12,7 +12,7 @@ Git Sources turn any stack into a GitOps target. Point Sencho at a repository an
## How it works
1. Open a stack and click the **Git Source** button in the editor toolbar.
2. Fill in the repository URL and branch, then choose the compose files. Use **Browse** to pick them from the repository tree, or type a path and press Enter. Add a token if the repo is private.
2. Fill in the repository URL and ref, then choose the compose files. Use **Browse** to pick them from the repository tree, or type a path and press Enter. Add a token if the repo is private.
3. Click **Pull now** to fetch the latest commit. Sencho opens a classified change plan: adds, modifications, removals, and any local conflicts.
4. Click **Apply** to write the incoming files to disk. Apply stays disabled while local file conflicts are present. A Compose invocation change (for example a `.env` file added or removed outside Git) is shown in the plan and does not disable Apply. Applying records the incoming invocation as the new baseline and leaves unmanaged files on disk. Tick **Deploy after apply** in the same dialog to redeploy in one step.
@@ -21,15 +21,15 @@ Writes land in the stack's existing directory using the same storage Sencho uses
## Anatomy of the panel
<Frame>
<img src="/images/git-sources/panel.png" alt="Git Source panel for a stack already linked to a repository, showing populated Repository URL, Branch, Compose file path, the Authentication toggle, the Apply behavior radio group, and a Last applied commit row at the bottom" />
<img src="/images/git-sources/panel.png" alt="Git Source panel for a stack already linked to a repository, showing populated Repository URL, Ref, Compose file path, the Authentication toggle, the Apply behavior radio group, and a Last applied commit row at the bottom" />
</Frame>
The panel groups four regions:
- **Pending update banner.** Appears at the top whenever a fetched commit is staged, however it was fetched. Its heading is the source state, so it says whether the commit is ready to apply, waiting on review, or blocked by local conflicts. Click **Review** to re-fetch the incoming commit and open the change plan.
- **Form fields.** Repository URL, branch, the ordered compose-file picker, an optional project directory, optional sibling `.env` sync, authentication toggle, and the apply behavior radio group.
- **Form fields.** Repository URL, ref, the ordered compose-file picker, an optional project directory, optional sibling `.env` sync, authentication toggle, and the apply behavior radio group.
- **Last applied stat strip.** Shows the short SHA of the last commit Sencho applied to disk, the source state (see below), and the timestamp of the most recent successful save or pull.
- **Footer actions.** **Remove** disconnects the source by exporting the effective compose model into a single `compose.yaml` and removing auto-discovered override files; the remaining materialized files are kept. **Pull now** fetches the configured branch's HEAD; **Save** or **Update** persists form changes after a reachability check passes.
- **Footer actions.** **Remove** disconnects the source by exporting the effective compose model into a single `compose.yaml` and removing auto-discovered override files; the remaining materialized files are kept. **Pull now** fetches the configured branch, tag, or commit's current revision; **Save** or **Update** persists form changes after a reachability check passes.
## Source state
@@ -55,7 +55,7 @@ These are the states you will see most often.
The last two matter most. Sencho reports an interrupted operation as unknown rather than guessing, so a stack whose pull was cut short by a restart says so instead of quietly reading as up to date.
<Note>
Changing the repository, the branch, the compose files, the project directory, or the `.env` sync clears any staged commit, because the plan was built against the settings you just replaced. Pull again to rebuild it. The state moves to **Reconcile required** if a commit had already been accepted, and to **Never reconciled** if none ever was. Changing only the token or the apply behavior leaves a staged commit alone, since neither changes what would be materialized.
Changing the repository, the ref, the compose files, the project directory, or the `.env` sync clears any staged commit, because the plan was built against the settings you just replaced. Pull again to rebuild it. The state moves to **Reconcile required** if a commit had already been accepted, and to **Never reconciled** if none ever was. Changing only the token or the apply behavior leaves a staged commit alone, since neither changes what would be materialized.
</Note>
### When part of the state could not be proven
@@ -90,14 +90,14 @@ Tick **Deploy after create** to run `docker compose up -d` immediately after the
| Field | Description |
|-------|-------------|
| **Repository URL** | `https://github.com/your-org/your-repo.git` (HTTPS only) |
| **Branch** | Branch to track (e.g. `main`) |
| **Ref** | Branch, tag, or full commit SHA to track (e.g. `main`, `v1.0`, or a 40-character SHA) |
| **Compose files** | One or more paths within the repo, merged in the listed order (e.g. `deploy/base.yaml` then `deploy/prod.yaml`). The first file is the primary. Reorder by dragging (or the up/down arrows on a phone) and remove with the **×** button. |
| **Project directory** | Optional path within the repo passed to `docker compose --project-directory`, so relative build contexts, bind mounts, and `env_file` references resolve from that base. Leave blank to use the stack root. |
| **Also sync sibling `.env` file** | When enabled, also pulls the `.env` from the same directory as the primary compose file. The form shows the resolved path inline (e.g. `deploy/.env` for a primary at `deploy/compose.yaml`). |
| **Authentication** | **Public (no auth)** for public repos, **Personal Access Token** for private repos |
| **Apply behavior** | See the three modes below |
Saving runs a reachability check against the repository. If the URL is wrong, the token is invalid, the branch does not exist, or a file is missing, Sencho surfaces the error inline and nothing is persisted.
Saving runs a reachability check against the repository. If the URL is wrong, the token is invalid, the ref does not exist, or a file is missing, Sencho surfaces the error inline and nothing is persisted.
### Multiple compose files
@@ -119,7 +119,7 @@ You can always override on the spot: when you click **Apply** in the change plan
## Pulling and reviewing changes
Click **Pull now** on the Git Source panel to fetch the latest commit on the configured branch.
Click **Pull now** on the Git Source panel to fetch the latest commit on the configured branch, tag, or commit.
<Frame>
<img src="/images/git-sources/diff-dialog.png" alt="GIT · CHANGE PLAN dialog for the demo-app stack, listing classified file operations (add, modify, remove) with a Deploy after apply checkbox and an Apply button in the footer" />
@@ -217,8 +217,16 @@ Pulls, applies, and create-from-git operations on the same stack are serialized
You supplied a token and the Git host rejected it outright. The token is missing, expired, or lacks read access. Generate a new token and replace the value in the **Token** field. Sencho returns this as a 400 form error rather than a 401, so an upstream auth failure does not sign you out of the dashboard.
</Accordion>
<Accordion title="Branch not found">
The branch name is case-sensitive and must exist on the remote. Confirm the branch with `git ls-remote <url>` from a shell that has access.
<Accordion title="Branch or tag not found">
The configured branch or tag is case-sensitive and must exist on the remote. Confirm it with `git ls-remote <url>` from a shell that has access. Sencho resolves the ref before fetching, so a typo or a ref that was never pushed surfaces here rather than as a generic failure. A full commit SHA never produces this error: it resolves to itself, so a SHA the host refuses to serve reports as a host-capability problem under "Commit not reachable on this host".
</Accordion>
<Accordion title="Branch or tag deleted or force-pushed">
The configured ref previously resolved to a commit but no longer matches that history. The ref may have been deleted, renamed, superseded by a same-named tag, or force-pushed to a history the old commit is no longer part of. Point the source at a current branch, tag, or commit, or restore the ref upstream, then save again.
</Accordion>
<Accordion title="Commit not reachable on this host">
The configured commit SHA is not one the Git host will serve. Hosts only fetch SHAs they advertise by default, so a commit that is not on any branch or tag tip, or one on a host that blocks unadvertised object fetch, returns this. Use a branch or tag, or a commit the host advertises.
</Accordion>
<Accordion title="File not found">
@@ -283,7 +291,7 @@ Pulls, applies, and create-from-git operations on the same stack are serialized
- **HTTPS only.** SSH URLs and SSH keys are not supported. Use a Personal Access Token for private repos.
- **No Git LFS.** Compose and env files stored via LFS are rejected. Commit plain files instead.
- **No submodules.** Submodule contents are not fetched. Inputs and build contexts that reference submodule contents are refused with an actionable message; a warning is shown when `.gitmodules` is present.
- **Branch-tracking only.** Sources follow the head of a branch. Specific commit SHAs and tags are not pinnable. Each pull resolves and pins the exact commit SHA, so apply always materializes the reviewed revision.
- **Refs, not arbitrary commits.** Sources follow a branch head, a tag, or a pinned commit SHA. Each pull resolves the configured ref to the exact commit it currently points at and pins that SHA, so apply always materializes the reviewed revision. A commit SHA not advertised by the Git host is refused.
- **Clone size cap.** A clone is bounded on the on-disk size of its temporary workspace (and each compose/env file is capped on read), so very large repositories are rejected. Operators can adjust the workspace ceiling with `GITSOURCE_MAX_CLONE_BYTES`.
- **Complete project materialization.** Every repository-local input the project needs is materialized: the ordered compose files, implicit `compose.override.*` files, recursive `include:` and `extends.file` dependencies, service env files, file-backed configs and secrets, label files, and build contexts with `.dockerignore` semantics. The materialized set is recorded in a versioned managed-project manifest, and each pull stages a candidate that is validated with the exact deployment invocation before anything on disk changes. If apply is interrupted, Sencho completes the accepted generation or restores the previous generation. If files were edited during the interruption and no longer match either generation, Sencho preserves them and requires manual recovery instead of overwriting them.
- **Unsupported inputs are refused, not guessed.** Inputs that cannot be safely reproduced fail the pull with an actionable message: URL includes, Git LFS pointers, submodule contents, symbolic links, build contexts that exceed the size bounds, and include or extends declarations that point outside the repository or use dynamic `\${VAR}` paths (their contents cannot be enumerated). Nothing is applied until the declaration is fixed. Absolute host paths, host bind mounts, external resources, and dynamic `\${VAR}` data paths are never claimed as covered: they resolve at deploy time from the environment or the node, and the manifest records them as unmanaged.
+6 -6
View File
@@ -31,11 +31,11 @@ This tutorial covers linking an existing stack to a Git source and running one m
Open the `marketing-site` stack and select **Git Source** in the editor toolbar. The panel opens empty, since nothing is linked yet.
<Frame>
<img src="/images/tutorials/connect-a-git-source/git-source-panel-empty.png" alt="Empty Git source panel for the marketing-site stack, showing blank Repository URL and Branch fields defaulted to main, a compose.yaml entry marked primary in Compose files, Public (no auth) selected under Authentication, and Review only selected under Apply behavior." />
<img src="/images/tutorials/connect-a-git-source/git-source-panel-empty.png" alt="Empty Git source panel for the marketing-site stack, showing blank Repository URL and Ref fields defaulted to main, a compose.yaml entry marked primary in Compose files, Public (no auth) selected under Authentication, and Review only selected under Apply behavior." />
</Frame>
</Step>
<Step title="Point it at your repository">
Paste your repository's HTTPS URL into **Repository URL** (for example `https://github.com/your-org/your-repo.git`). Leave **Branch** on its default, `main`, unless your repository uses a different one.
Paste your repository's HTTPS URL into **Repository URL** (for example `https://github.com/your-org/your-repo.git`). Leave **Ref** on its default, `main`, unless your repository uses a different branch, tag, or pinned commit SHA.
Select **Browse** to confirm Sencho can actually reach the repository and see its files, rather than trusting the URL is correct. The browser lists every file in the repository; `compose.yaml` is already ticked as the primary compose file, since that name matches the picker's default.
@@ -46,9 +46,9 @@ This tutorial covers linking an existing stack to a Git source and running one m
Leave **Authentication** on **Public (no auth)** for a public repository, and **Apply behavior** on **Review only**, the safest default: a pull only stages a change plan for you to review, it never writes or deploys on its own. Select **Save**. Sencho runs a reachability check against the repository before persisting anything; if that check fails, nothing is saved and the panel reports why.
</Step>
<Step title="Pull the latest commit">
Now make a change the way your team actually would: edit the compose file in your repository (not in Sencho) and push a commit. For this tutorial, bump the pinned tag from `nginx:1.27-alpine` to `nginx:1.28-alpine` and push it to the branch you configured.
Now make a change the way your team actually would: edit the compose file in your repository (not in Sencho) and push a commit. For this tutorial, bump the pinned tag from `nginx:1.27-alpine` to `nginx:1.28-alpine` and push it to the ref you configured.
Back in the Git Source panel, select **Pull now**. Sencho fetches the branch's current commit and opens a classified change plan against what's on disk.
Back in the Git Source panel, select **Pull now**. Sencho fetches the ref's current commit and opens a classified change plan against what's on disk.
<Frame>
<img src="/images/tutorials/connect-a-git-source/pull-preview-diff.png" alt="GIT · CHANGE PLAN dialog for marketing-site, listing a Modify row for compose.yaml (image tag change) with Deploy after apply and Apply in the footer." />
@@ -81,10 +81,10 @@ Check from two places, since neither alone proves the pull was actually applied
## If something goes wrong
**Saving the source fails with "Repository not found or not accessible," even though the URL is right.** Branch names are case-sensitive, and a typo there (`Main` instead of `main`, for example) surfaces as this same repository-level error rather than a distinct branch error, since Sencho can't always tell a missing branch apart from a missing repository during the reachability check. Double-check the branch name against what your Git host actually shows before assuming the URL itself is wrong.
**Saving the source fails with "The configured branch, tag, or commit was not found," even though the URL is right.** Ref names are case-sensitive, and a typo there (`Main` instead of `main`, for example) surfaces as a distinct not-found error rather than a repository error, because Sencho resolves the ref on the remote before fetching. Double-check the ref against what your Git host actually shows before assuming the URL itself is wrong.
<Frame>
<img src="/images/tutorials/connect-a-git-source/branch-error.png" alt="Git source panel with Branch set to the incorrect value Main, and a red error toast in the corner reading Repository not found or not accessible." />
<img src="/images/tutorials/connect-a-git-source/branch-error.png" alt="Git source panel with Ref set to the incorrect value Main, and a red error toast in the corner reading The configured branch, tag, or commit was not found." />
</Frame>
## Related
@@ -130,10 +130,10 @@ export function GitSourceFields({
</div>
<div className="space-y-2">
<Label htmlFor="git-source-branch">Branch</Label>
<Label htmlFor="git-source-branch">Ref</Label>
<Input
id="git-source-branch"
placeholder="main"
placeholder="main, v1.0, or commit SHA"
value={branch}
onChange={(e) => onBranchChange(e.target.value)}
disabled={disabled}
@@ -198,7 +198,7 @@ export function GitSourcePanel({
const save = async () => {
if (!repoUrl.trim() || !branch.trim() || composePaths.length === 0) {
toast.error('Repository URL, branch, and at least one compose file are required.');
toast.error('Repository URL, ref, and at least one compose file are required.');
return;
}
if (!/^https:\/\//i.test(repoUrl.trim())) {
@@ -253,7 +253,7 @@ export function GitSourcePanel({
const browseRepo = async (): Promise<GitBrowseResult | null> => {
if (!repoUrl.trim() || !branch.trim()) {
toast.error('Enter a repository URL and branch first.');
toast.error('Enter a repository URL and ref first.');
return null;
}
try {