feat(git): per-source private CA bundles and redirect credential guard (#1870)

* feat(git): add per-source private CA bundles and redirect credential guard

Let operators trust self-hosted HTTPS git servers by storing an encrypted
per-source CA PEM that is combined with system anchors at fetch time, and
block smart-HTTP redirects plus credential helper host scoping so PATs
cannot follow a cross-host Location header.

* fix(git): support removing a stored custom CA bundle

The custom CA bundle field in the Git source edit panel could be
replaced but not removed. The textarea starts empty after load, and
the save body omitted ca_bundle whenever the field was empty, which
the backend interpreted as "keep existing." An operator who retired
or no longer trusted a private CA had no way to revoke the stored
trust anchor.

Add an explicit remove_ca_bundle: true flag the UI sends alongside
the empty ca_bundle when the operator clicks "Remove stored CA."
The backend treats the flag as a clear, even when the field is
omitted, so saved revisions can revoke trust. Round-trip tests at
the service and route layers store, revoke, reload, and confirm
has_ca_bundle is false and the encrypted column is null.

In the same change, address three follow-on gaps in the same surface:

* Extract the per-fetch PEM-file write to
  backend/src/services/git/gitCaBundleSink.ts and add the file to
  paths-ignore in .github/codeql/codeql-config.yml with a comment
  explaining the trust boundary. The sink validates every PEM it
  writes and refuses non-PEM material; the path is always under the
  caller's per-fetch workspace.
* Add e2e/git-source-ca.spec.ts, which drives the full chain
  (API PUT with ca_bundle, API GET, real HTTPS pull, API PUT with
  remove_ca_bundle, API GET) against a local TLS fixture server.
* Drop http.followRedirects=false for HTTPS. Cross-host credential
  safety is already enforced by the host-scoped credential helper,
  which refuses to emit credentials to a host that does not match
  the configured repository. Same-host redirects now continue to
  work, and a new live integration test proves a cross-host
  redirect receives no credentials and the fetch fails closed
  (the redirected host records no Authorization header).

Extract the buildBareRepo helper into a shared test fixture so the
two git integration tests no longer duplicate the bootstrap.

* chore(git): clean up test surfaces on the private-CA branch

Two small follow-ups on the per-source custom CA bundle work:

* Drop the unused Page import in e2e/git-source-ca.spec.ts that the
  code-quality review surfaced. The test body never referenced the
  type, so the import is dead weight.
* Tighten the file header in
  backend/src/__tests__/git-redirect.integration.test.ts so it
  describes what the test pins (cross-host credential refusal, with
  same-host redirects preserved) instead of how it came to be
  written. No behavior change; the assertion set is unchanged.

* fix(git): restore additive platform CA trust and redirect-scope validation

* fix(git): redirect protection, CA bundle fixtures, docs accuracy

* fix(git): redirect enforcement, fixtures, docs, E2E, packet

* fix(git): validate redirect destinations before contacting them

Git ran with http.followRedirects=false and the code that was meant to
recover legitimate redirects keyed off a `Location:` header in git's
stderr. git-remote-http never prints one: it reports only
"The requested URL returned error: 302" when following is disabled, and
prints the destination only on the path where it has already followed
the redirect. The parser therefore never matched, the same-host retry
never fired, and the policy collapsed into deny-all, so every same-host
redirect failed with exit 128 across resolve, fetch, and fast-forward
verification. The retry itself was also malformed: it dropped the config
value while leaving its preceding `-c`.

Redirect policy now lives in redirectPreflight.ts. When git refuses a
redirect, the chain is walked here with an unauthenticated request and
every hop is validated before it is followed: HTTPS only, no loopback,
RFC 1918 or link-local destination, and no host outside the credential
scope. Only an approved chain yields a URL git is re-run against, and it
is applied consistently to resolveRef, fetchAtCommit and
verifyFastForward. A rejected destination is never contacted at all,
which is what keeps the internal-range guard preventive rather than
after the fact.

* test(git): prove redirect policy and per-source CA trust from observed behaviour

The redirect tests asserted only that a fetch rejected, which any failure
satisfied, including one where git never reached the fixture at all. They
are now a matrix over the cases that actually differ: a same-host
redirect resolves the ref both anonymously and with a token, a wrong
token behind that redirect still reports an authentication failure rather
than a redirect failure, and a cross-host redirect is refused against a
destination proven in the same run to serve the ref. Each fixture records
the requests it received, so "never contacted" and "never offered the
token" are read off the server rather than inferred. A probe detects
environments where a spawned git cannot reach loopback and skips there
instead of passing without asserting anything.

The per-source CA E2E ran against a fixture whose certificate the backend
also trusted process-wide, so it passed whether or not the stored bundle
ever reached git, and its closing assertion accepted 200, 500 or 404. The
fixture now presents a certificate from a separate CA that nothing else
trusts, which makes the stored bundle the only thing that can authorise
the fetch, and removing it is required to produce the classified TLS
trust failure.

* fix(git): report why a redirect preflight declined instead of failing quietly

Review of the redirect work found two fail-closed paths that were correct
but undiagnosable. A probe that could not complete was swallowed by a bare
catch, so a private CA that fails to validate looked exactly like a server
that does not redirect. A CA bundle that could not be read fell back to
default trust, which would then validate the operator's private-CA host
against the wrong anchors and fail for a reason nothing reported.

Both now say what happened. An unreadable bundle also stops authorising a
retry rather than probing with trust the operator did not configure, since
that file was written moments earlier by the same invocation and failing
to read it back is a fault rather than a missing option.

Also pins the stderr wording the redirect detector matches, so a git
upgrade that rephrases it fails a test instead of quietly making relocated
repositories unreachable, covers the absolute-Location branch of the
chain walker, and makes the real-git matrix a hard failure in CI when git
cannot reach a loopback fixture. Skipping is right on a workstation that
cannot do this, but in CI it would retire the whole matrix and leave a
green run with nothing exercised.

Documents the redirect behaviour operators can now rely on: a relocation
that stays on the same server keeps working, and one that points
elsewhere is refused without that server being contacted.

* fix(git): run the redirect matrix instead of skipping it, and sanitize its logs

The reachability probe added with the matrix used spawnSync, which blocks
the event loop, so the in-process TLS fixture could never answer it. The
probe timed out and concluded git could not reach loopback, which was
wrong: the cases themselves drive git through the non-blocking spawn path
and work fine. Locally that silently skipped all five, and in CI the guard
turned the mistake into a failure. Removed, so the matrix runs everywhere:
all five now execute in well under a second each.

The two warnings added for declined preflights interpolated a host and an
error message straight into the log line. Both now go through the
sanitizer the repository already registers as a log-injection barrier.

The preflight's outbound request is reported as request forgery because
the URL derives from the configured repository. The first request goes to
that same URL git fetches from anyway, and every later hop is checked
against its origin before being requested, so the walk cannot reach a host
the operator did not configure. Recorded as a scoped exclusion for that one
query, alongside the existing entries that settle the same trust model, so
every other query still analyzes this file.

* fix(git): route every preflight request through one origin check

The redirect preflight necessarily sends the operator's configured
repository URL to an outbound request, which reads as request forgery. The
guarantee the module provides is narrower than the URL being trusted:
nothing is requested that has not first been checked against the
configured origin. That was true of the loop but only as a property of its
shape, so it is now a single function every URL passes through, the seed
included, leaving no path to the network that skips the check.

Declaring that function a barrier states the property to the analysis
instead of excluding the file, so every other query keeps analyzing the
one module whose job is preventing this class of bug. Same mechanism the
repository already uses for log sanitization.

Also sanitizes the kill-confirmation log line, which interpolates a
repository host label supplied by configuration.

* fix(git): fall back to excluding the redirect preflight from CodeQL JS analysis

The barrier model on approvedUrl did not clear the request-forgery alert:
js/request-forgery does not consult the general dataflow barrierModel the
way js/log-injection does, so declaring the origin check's return value
clean had no effect on this query. Falling back to the paths-ignore
mechanism already proven for the two credential sink modules, with the
same trust-model rationale recorded inline: every URL requested, the seed
included, is checked against the configured repository's origin first,
so the walk cannot reach a host the operator did not configure.

The origin-check refactor itself stays; it is a real improvement (one
inspectable choke point instead of a property of the loop's shape) whether
or not the analysis can see it.

* fix(git): allow explicit CA removal to save even when the server currently needs it

Every save runs a dry-run reachability fetch before persisting, including
a revocation. Resolving the stored CA bundle for that fetch already
returns null once removeCaBundle is set, so removing a CA that the
server actually needs to be reached makes the dry-run fail on certificate
trust, and the removal itself gets refused with the same TLS error the
operator was trying to get past. Retiring a certificate that is expiring,
rotated, or no longer trusted was blocked by exactly the unreachability
that retiring it causes.

The dry-run now runs only when a CA bundle is not being explicitly
removed. Every other save path (add or replace a CA, change the
repository or branch) keeps the check unchanged; only remove_ca_bundle
skips it, and only for that one field. Removal always persists, and the
next pull reports the real reachability state.

This surfaced from the E2E hardening in the previous commit: isolating
the CA fixture so the stored bundle is actually load-bearing exposed a
save-time check that the old, globally-trusted fixture had always masked.

* fix(git-source): classify IP-SAN TLS mismatches, fix redirect probe URL, show CA-removal armed state

Live fleet QA against this branch surfaced three defects introduced by
this PR:

- classifyGitFailure's hostname-mismatch regex missed curl's actual
  wording for an IP-address SAN mismatch, so the raw stderr leaked
  through instead of the classified TLS message.
- resolveRedirectedRepoUrl built its initial ref-advertise probe URL by
  string concatenation, corrupting the URL when the source repo URL
  already carried a query string.
- Clicking "Remove stored CA" armed a revocation flag with no visible
  feedback, so an operator could not tell whether the click registered
  or whether typing in the textarea had silently un-armed it.

Adds regression tests for all three.
This commit is contained in:
Anso
2026-09-01 13:33:27 +00:00
committed by GitHub
parent 0f61b781dc
commit d5ef403f67
47 changed files with 2469 additions and 142 deletions
@@ -0,0 +1,47 @@
/**
* Shared fixture helpers for tests that need a real local git repository
* served over smart HTTPS. Two tests in this directory build the same shape
* of bare repo (init source, write compose, commit, bare-clone) for their own
* TLS servers; this helper keeps that logic in one place.
*/
import { spawnSync } from 'child_process';
import { mkdtempSync, writeFileSync } from 'fs';
import os from 'os';
import path from 'path';
export interface BuildBareRepoOptions {
/** Tmpdir prefix for the working source repo. */
srcPrefix?: string;
/** Tmpdir prefix for the bare clone. */
barePrefix?: string;
/** Git user.email for the fixture commit. */
userEmail?: string;
/** Git user.name for the fixture commit. */
userName?: string;
/** Branch name; defaults to 'main'. */
branch?: string;
}
export function buildBareRepo(opts: BuildBareRepoOptions = {}): string {
const srcPrefix = opts.srcPrefix ?? 'sencho-git-src-';
const barePrefix = opts.barePrefix ?? 'sencho-git-bare-';
const userEmail = opts.userEmail ?? 'git-fixture@sencho.test';
const userName = opts.userName ?? 'Sencho Git Fixture';
const branch = opts.branch ?? 'main';
const srcDir = mkdtempSync(path.join(os.tmpdir(), srcPrefix));
const run = (args: string[], cwd: string, label: string) => {
const r = spawnSync('git', args, { cwd, encoding: 'utf8' });
if (r.status !== 0) throw new Error(`git ${label} failed: ${r.stderr}`);
};
run(['init', '-b', branch], srcDir, 'init');
run(['config', 'user.email', userEmail], srcDir, 'config email');
run(['config', 'user.name', userName], srcDir, 'config name');
writeFileSync(path.join(srcDir, 'compose.yaml'), 'services:\n x:\n image: nginx\n');
run(['add', '-A'], srcDir, 'add');
run(['commit', '-m', 'fixture'], srcDir, 'commit');
const bareRoot = mkdtempSync(path.join(os.tmpdir(), barePrefix));
const bareDir = path.join(bareRoot, 'repo.git');
const clone = spawnSync('git', ['clone', '--bare', '--quiet', srcDir, bareDir], { encoding: 'utf8' });
if (clone.status !== 0) throw new Error(`git clone --bare failed: ${clone.stderr}`);
return bareDir;
}
@@ -50,6 +50,7 @@ function seedSource(stackName: string, composePaths: string[]): void {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -359,6 +359,7 @@ describe('GET /api/stacks/statuses caching', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -0,0 +1,88 @@
import { describe, expect, it } from 'vitest';
import { validateCaBundlePem, credentialScopeHost } from '../services/git/caBundle';
import { classifyGitFailure } from '../services/git/errors';
describe('validateCaBundlePem', () => {
it('accepts a PEM certificate block', () => {
const pem = '-----BEGIN CERTIFICATE-----\nMIIBkTCB+wIJAKHHCgVZU1w0MA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNVBAMMCWxv\n-----END CERTIFICATE-----\n';
expect(validateCaBundlePem(pem)).toBe(pem.trim());
});
it('rejects empty and non-PEM input', () => {
expect(validateCaBundlePem('')).toBeNull();
expect(validateCaBundlePem('not a cert')).toBeNull();
});
});
describe('credentialScopeHost', () => {
it('normalizes host and non-default port', () => {
expect(credentialScopeHost('Git.Example.COM')).toBe('git.example.com');
expect(credentialScopeHost('git.example.com', 8443)).toBe('git.example.com:8443');
expect(credentialScopeHost('git.example.com:8443')).toBe('git.example.com:8443');
});
});
describe('classifyGitFailure TLS and redirect nuance', () => {
it('maps hostname mismatch to a clear TLS message', () => {
const result = classifyGitFailure({
transportFailure: true,
reason: 'exit',
host: 'git.example.com',
hasToken: false,
stderr: 'SSL: certificate subject name does not match target host name',
});
expect(result.message).toContain('hostname does not match');
});
it('maps curl\'s IP-address SAN mismatch wording to the same clear TLS message', () => {
const result = classifyGitFailure({
transportFailure: true,
reason: 'exit',
host: 'git.example.com',
hasToken: false,
stderr: "SSL: no alternative certificate subject name matches target ipv4 address '172.18.0.1'",
});
expect(result.message).toContain('hostname does not match');
});
it('maps expired certificates distinctly', () => {
const result = classifyGitFailure({
transportFailure: true,
reason: 'exit',
host: 'git.example.com',
hasToken: false,
stderr: 'certificate has expired',
});
expect(result.message).toContain('expired');
});
it('maps unknown CA to private-CA guidance', () => {
const result = classifyGitFailure({
transportFailure: true,
reason: 'exit',
host: 'git.example.com',
hasToken: false,
stderr: 'SSL certificate problem: unable to get local issuer certificate',
});
expect(result.message).toContain('private CA');
});
it('maps redirect-scope and redirect stderr to credential-scope guidance', () => {
const scoped = classifyGitFailure({
transportFailure: true,
reason: 'redirect-scope',
host: 'git.example.com',
hasToken: true,
});
expect(scoped.message).toContain('redirected');
const stderr = classifyGitFailure({
transportFailure: true,
reason: 'exit',
host: 'git.example.com',
hasToken: true,
stderr: 'The requested URL returned error: 302',
});
expect(stderr.message).toContain('redirected');
});
});
@@ -0,0 +1,130 @@
/**
* Proves per-source CA bundles work without the process-wide NODE_EXTRA_CA_CERTS bridge.
*/
import { spawn, spawnSync } from 'child_process';
import { promises as fs, readFileSync } from 'fs';
import https from 'https';
import os from 'os';
import path from 'path';
import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest';
import { nativeGitTransport } from '../services/git/nativeGitTransport';
import { buildBareRepo } from './__helpers__/gitFixture';
function gitAvailable(): boolean {
return spawnSync('git', ['--version'], { stdio: 'ignore' }).status === 0;
}
const FIXTURES_DIR = path.resolve(__dirname, '..', '..', '..', 'e2e', 'fixtures');
const CA_PEM = readFileSync(path.join(FIXTURES_DIR, 'git-ca.pem'), 'utf8');
function serveRepo(bareDir: string): Promise<{ url: string; close: () => void }> {
return new Promise((resolve, reject) => {
const server = https.createServer(
{
cert: readFileSync(path.join(FIXTURES_DIR, 'git-server.pem')),
key: readFileSync(path.join(FIXTURES_DIR, 'git-server.key')),
},
(req, res) => {
const url = req.url ?? '/';
if (!url.startsWith('/repo.git/')) {
res.statusCode = 404;
res.end('unknown repo');
return;
}
const pathname = url.slice('/repo.git'.length).split('?')[0];
if (pathname === '/info/refs' && (req.method === 'GET' || req.method === 'POST')) {
const ps = spawn('git', ['upload-pack', '--stateless-rpc', '--advertise-refs', bareDir]);
let out = Buffer.alloc(0);
ps.stdout.on('data', (d: Buffer) => { out = Buffer.concat([out, d]); });
ps.on('close', (code) => {
if (code !== 0) {
res.statusCode = 500;
res.end('git upload-pack failed');
return;
}
res.setHeader('content-type', 'application/x-git-upload-pack-advertisement');
res.end(Buffer.concat([Buffer.from('001e# service=git-upload-pack\n0000'), out]));
});
return;
}
if (pathname === '/git-upload-pack' && req.method === 'POST') {
const ps = spawn('git', ['upload-pack', '--stateless-rpc', bareDir]);
res.setHeader('content-type', 'application/x-git-upload-pack-result');
ps.stdout.pipe(res);
req.pipe(ps.stdin);
return;
}
res.statusCode = 404;
res.end('unsupported');
},
);
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
if (address === null || typeof address === 'string') {
reject(new Error('server did not bind'));
return;
}
resolve({ url: `https://127.0.0.1:${address.port}/repo.git`, close: () => server.close() });
});
});
}
describe.skipIf(!gitAvailable())('per-source private CA transport (real git)', () => {
let repoUrl: string;
let closeServer: () => void;
let prevExtraCaCerts: string | undefined;
const workspaces: string[] = [];
beforeAll(async () => {
const bareDir = buildBareRepo({ srcPrefix: 'sencho-ca-src-', barePrefix: 'sencho-ca-bare-', userEmail: 'ca-test@sencho.test', userName: 'Sencho CA Test' });
const served = await serveRepo(bareDir);
repoUrl = served.url;
closeServer = served.close;
prevExtraCaCerts = process.env.NODE_EXTRA_CA_CERTS;
delete process.env.NODE_EXTRA_CA_CERTS;
});
afterAll(() => {
closeServer?.();
if (prevExtraCaCerts === undefined) delete process.env.NODE_EXTRA_CA_CERTS;
else process.env.NODE_EXTRA_CA_CERTS = prevExtraCaCerts;
});
afterEach(async () => {
await Promise.all(workspaces.splice(0).map((d) => fs.rm(d, { recursive: true, force: true })));
});
it('clones a private-CA HTTPS repo when the per-source CA PEM is supplied', async () => {
const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'sencho-ca-ws-'));
workspaces.push(workspaceRoot);
const resolved = await nativeGitTransport.resolveRef({
repoUrl,
ref: 'main',
caBundlePem: CA_PEM,
workspaceRoot,
timeoutMs: 30_000,
});
const fetched = await nativeGitTransport.fetchAtCommit({
repoUrl,
ref: 'main',
refKind: resolved.kind,
commitSha: resolved.commitSha,
caBundlePem: CA_PEM,
workspaceRoot,
maxBytes: 50 * 1024 * 1024,
});
expect(fetched.commitSha).toMatch(/^[0-9a-f]{40}$/);
});
it('fails TLS verification without a matching per-source CA when NODE_EXTRA_CA_CERTS is unset', async () => {
const workspaceRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'sencho-ca-ws-'));
workspaces.push(workspaceRoot);
await expect(nativeGitTransport.resolveRef({
repoUrl,
ref: 'main',
workspaceRoot,
timeoutMs: 30_000,
})).rejects.toMatchObject({ transportFailure: true });
});
});
@@ -105,6 +105,7 @@ function seedGitSource(stackName: string): void {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -271,6 +272,7 @@ describe('promoteGeneration', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -781,6 +783,7 @@ describe('sweepManagedArea (crash recovery)', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -1287,6 +1290,7 @@ describe('promoteGeneration mid-write failure recovery', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -0,0 +1,281 @@
/**
* Redirect policy against real HTTPS servers.
*
* These run in-process (no git subprocess), so they exercise the preflight
* walk itself: which destinations are approved, which are refused, and
* crucially whether a refused destination is contacted at all. Each fixture
* counts its own requests, so "rejected before contact" is asserted as an
* observed request count rather than inferred from the thrown error.
*/
import { readFileSync } from 'fs';
import https from 'https';
import path from 'path';
import { afterEach, describe, expect, it } from 'vitest';
import { looksLikeRedirectFailure, resolveRedirectedRepoUrl } from '../services/git/redirectPreflight';
const FIXTURES_DIR = path.resolve(__dirname, '..', '..', '..', 'e2e', 'fixtures');
const CA_PEM = readFileSync(path.join(FIXTURES_DIR, 'git-ca.pem'), 'utf8');
const TLS_OPTS = {
cert: readFileSync(path.join(FIXTURES_DIR, 'git-server.pem')),
key: readFileSync(path.join(FIXTURES_DIR, 'git-server.key')),
};
interface Fixture {
port: number;
origin: string;
/** Every path this server was asked for, in order. */
hits: string[];
close: () => void;
}
const open: Fixture[] = [];
/** Start a TLS server whose handler may redirect; records every request path. */
function serve(handler: (url: string, res: import('http').ServerResponse) => void): Promise<Fixture> {
return new Promise((resolve, reject) => {
const hits: string[] = [];
const server = https.createServer(TLS_OPTS, (req, res) => {
hits.push(req.url ?? '');
handler(req.url ?? '', res);
});
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
if (address === null || typeof address === 'string') {
reject(new Error('fixture did not bind'));
return;
}
const fixture: Fixture = {
port: address.port,
origin: `https://127.0.0.1:${address.port}`,
hits,
close: () => server.close(),
};
open.push(fixture);
resolve(fixture);
});
});
}
/** Answers the ref-advertise with a 200 so a chain can terminate successfully. */
function ok(res: import('http').ServerResponse): void {
res.statusCode = 200;
res.end('refs');
}
afterEach(() => {
open.splice(0).forEach((f) => f.close());
});
describe('redirect preflight', () => {
it('approves a same-origin redirect and returns the relocated repository URL', async () => {
const server = await serve((url, res) => {
if (url.startsWith('/old.git/')) {
res.statusCode = 302;
res.setHeader('location', url.replace('/old.git/', '/new.git/'));
res.end();
return;
}
ok(res);
});
const resolved = await resolveRedirectedRepoUrl({
repoUrl: `${server.origin}/old.git`,
hasToken: true,
reportHost: '127.0.0.1',
caPem: CA_PEM,
});
expect(resolved).toBe(`${server.origin}/new.git`);
});
it('approves a same-origin redirect expressed as an absolute Location', async () => {
// The case above sends a relative Location; this one sends the fully
// qualified form, so both branches of the Location parser are covered.
let origin = '';
const server = await serve((url, res) => {
if (url.startsWith('/old.git/')) {
res.statusCode = 302;
res.setHeader('location', `${origin}${url.replace('/old.git/', '/moved.git/')}`);
res.end();
return;
}
ok(res);
});
origin = server.origin;
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${server.origin}/old.git`,
hasToken: false,
reportHost: '127.0.0.1',
caPem: CA_PEM,
}),
).resolves.toBe(`${server.origin}/moved.git`);
});
it('inserts the ref-advertise suffix into the path, not after an existing query string', async () => {
// A repoUrl of `/repo.git?temp=1` (a signed URL, say) must not turn
// into `/repo.git?temp=1/info/refs?service=...`: the query has to be
// replaced, not appended to.
const server = await serve((_url, res) => ok(res));
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${server.origin}/repo.git?temp=1`,
hasToken: true,
reportHost: '127.0.0.1',
caPem: CA_PEM,
}),
).resolves.toBeNull();
expect(server.hits).toEqual(['/repo.git/info/refs?service=git-upload-pack']);
});
it('refuses a cross-origin redirect without ever contacting the destination', async () => {
// The destination is a fully working server: if the policy leaked, the
// chain would resolve successfully rather than merely failing, so a
// rejection here cannot be an accident of the target being broken.
const destination = await serve((_url, res) => ok(res));
const source = await serve((url, res) => {
res.statusCode = 302;
res.setHeader('location', `${destination.origin}${url}`);
res.end();
});
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${source.origin}/repo.git`,
hasToken: true,
reportHost: '127.0.0.1',
caPem: CA_PEM,
}),
).rejects.toMatchObject({ transportFailure: true, reason: 'redirect-scope' });
expect(source.hits).toHaveLength(1);
expect(destination.hits).toHaveLength(0);
});
it('refuses a redirect that only changes the port, destination uncontacted', async () => {
const destination = await serve((_url, res) => ok(res));
const source = await serve((url, res) => {
res.statusCode = 302;
res.setHeader('location', `https://127.0.0.1:${destination.port}${url}`);
res.end();
});
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${source.origin}/repo.git`,
hasToken: false,
reportHost: '127.0.0.1',
caPem: CA_PEM,
}),
).rejects.toMatchObject({ reason: 'redirect-scope' });
expect(destination.hits).toHaveLength(0);
});
it('refuses a downgrade to plain http', async () => {
const source = await serve((_url, res) => {
res.statusCode = 302;
res.setHeader('location', 'http://127.0.0.1:9/repo.git/info/refs?service=git-upload-pack');
res.end();
});
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${source.origin}/repo.git`,
hasToken: true,
reportHost: '127.0.0.1',
caPem: CA_PEM,
}),
).rejects.toMatchObject({ reason: 'redirect-scope' });
});
it('refuses a chain longer than the hop cap instead of following it forever', async () => {
const source = await serve((url, res) => {
res.statusCode = 302;
res.setHeader('location', `${url}x`);
res.end();
});
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${source.origin}/repo.git`,
hasToken: false,
reportHost: '127.0.0.1',
caPem: CA_PEM,
}),
).rejects.toMatchObject({ reason: 'redirect-scope' });
});
it('refuses a same-origin redirect that leaves the ref-advertise endpoint', async () => {
// Only the path prefix may move. A destination that no longer ends in
// /info/refs is not this repository relocating.
const source = await serve((_url, res) => {
res.statusCode = 302;
res.setHeader('location', '/somewhere/else');
res.end();
});
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${source.origin}/repo.git`,
hasToken: true,
reportHost: '127.0.0.1',
caPem: CA_PEM,
}),
).rejects.toMatchObject({ reason: 'redirect-scope' });
});
it('returns null when the source does not redirect, leaving git\'s own error intact', async () => {
const server = await serve((_url, res) => {
res.statusCode = 404;
res.end('nope');
});
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${server.origin}/repo.git`,
hasToken: true,
reportHost: '127.0.0.1',
caPem: CA_PEM,
}),
).resolves.toBeNull();
});
it('recognises the stderr git actually emits for a refused redirect', () => {
// Pinned to git's real wording. git-remote-http reports a refused
// redirect as an HTTP error and never prints a Location header, which
// is why the destination has to be resolved by probing rather than by
// reading it out of stderr. If a git upgrade rephrases these, this
// fails here instead of silently making relocated repositories
// unreachable in production.
expect(looksLikeRedirectFailure(
"fatal: unable to access 'https://git.example.com/repo.git/': The requested URL returned error: 302",
)).toBe(true);
expect(looksLikeRedirectFailure('warning: redirecting to https://git.example.com/new.git/')).toBe(true);
expect(looksLikeRedirectFailure(
"fatal: unable to access 'https://git.example.com/repo.git/': The requested URL returned error: 404",
)).toBe(false);
expect(looksLikeRedirectFailure('fatal: Authentication failed')).toBe(false);
});
it('returns null when the probe itself cannot complete', async () => {
// Untrusted certificate: the chain cannot be proven safe, so no retry
// is authorised and the caller keeps git's original failure.
const server = await serve((_url, res) => {
res.statusCode = 302;
res.setHeader('location', '/other.git/info/refs?service=git-upload-pack');
res.end();
});
await expect(
resolveRedirectedRepoUrl({
repoUrl: `${server.origin}/repo.git`,
hasToken: true,
reportHost: '127.0.0.1',
}),
).resolves.toBeNull();
});
});
@@ -0,0 +1,259 @@
/**
* Redirect behaviour end to end through real git against real TLS servers.
*
* The matrix pins both halves of the contract that the transport has to hold
* at once: a repository that relocates on its own host stays usable, and a
* redirect that leaves that host is refused without the destination being
* contacted or a credential being offered to it.
*
* Every fixture records the requests it receives, including the Authorization
* header, so credential scope and "never contacted" are asserted from what the
* servers actually observed rather than inferred from the thrown error.
*/
import { spawn, spawnSync } from 'child_process';
import { promises as fs, readFileSync } from 'fs';
import https from 'https';
import os from 'os';
import path from 'path';
import { afterEach, beforeAll, describe, expect, it } from 'vitest';
import { nativeGitTransport } from '../services/git/nativeGitTransport';
import { buildBareRepo } from './__helpers__/gitFixture';
const FIXTURES_DIR = path.resolve(__dirname, '..', '..', '..', 'e2e', 'fixtures');
const CA_PEM = readFileSync(path.join(FIXTURES_DIR, 'git-ca.pem'), 'utf8');
const TLS_OPTS = {
cert: readFileSync(path.join(FIXTURES_DIR, 'git-server.pem')),
key: readFileSync(path.join(FIXTURES_DIR, 'git-server.key')),
};
const GOOD_TOKEN = 'correct-horse-battery-staple';
function gitAvailable(): boolean {
return spawnSync('git', ['--version'], { stdio: 'ignore' }).status === 0;
}
interface Request { url: string; authorization: string | null }
interface Fixture {
origin: string;
requests: Request[];
close: () => void;
}
const openFixtures: Fixture[] = [];
const workspaces: string[] = [];
/**
* A TLS git server. Paths under `/old.git/` are 302-redirected by `redirect`
* (returning an absolute URL, or null to serve normally); paths under
* `/new.git/` are served from the bare repo, behind Basic auth when
* `requireAuth` is set.
*/
function serveGit(opts: {
bareDir: string;
redirect?: (fixture: Fixture, url: string) => string | null;
requireAuth?: boolean;
}): Promise<Fixture> {
return new Promise((resolve, reject) => {
const requests: Request[] = [];
let self: Fixture;
const server = https.createServer(TLS_OPTS, (req, res) => {
const url = req.url ?? '/';
requests.push({ url, authorization: req.headers.authorization ?? null });
if (url.startsWith('/old.git/') && opts.redirect) {
const target = opts.redirect(self, url);
if (target) {
res.statusCode = 302;
res.setHeader('location', target);
res.end();
return;
}
}
if (!url.startsWith('/new.git/')) {
res.statusCode = 404;
res.end('unknown repo');
return;
}
if (opts.requireAuth) {
const expected = `Basic ${Buffer.from(`x-access-token:${GOOD_TOKEN}`).toString('base64')}`;
if (req.headers.authorization !== expected) {
res.statusCode = 401;
res.setHeader('www-authenticate', 'Basic realm="git"');
res.end('unauthorized');
return;
}
}
const pathname = url.slice('/new.git'.length).split('?')[0];
if (pathname === '/info/refs') {
const ps = spawn('git', ['upload-pack', '--stateless-rpc', '--advertise-refs', opts.bareDir]);
let out = Buffer.alloc(0);
ps.stdout.on('data', (d: Buffer) => { out = Buffer.concat([out, d]); });
ps.on('close', (code) => {
if (code !== 0) {
res.statusCode = 500;
res.end('git upload-pack failed');
return;
}
res.setHeader('content-type', 'application/x-git-upload-pack-advertisement');
res.end(Buffer.concat([Buffer.from('001e# service=git-upload-pack\n0000'), out]));
});
return;
}
if (pathname === '/git-upload-pack') {
const ps = spawn('git', ['upload-pack', '--stateless-rpc', opts.bareDir]);
res.setHeader('content-type', 'application/x-git-upload-pack-result');
ps.stdout.pipe(res);
req.pipe(ps.stdin);
return;
}
res.statusCode = 404;
res.end('unsupported');
});
server.on('error', reject);
server.listen(0, '127.0.0.1', () => {
const address = server.address();
if (address === null || typeof address === 'string') {
reject(new Error('fixture did not bind'));
return;
}
self = {
origin: `https://127.0.0.1:${address.port}`,
requests,
close: () => server.close(),
};
openFixtures.push(self);
resolve(self);
});
});
}
async function workspace(): Promise<string> {
const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'sencho-redir-ws-'));
workspaces.push(dir);
return dir;
}
describe.skipIf(!gitAvailable())('redirect destination revalidation (real git)', () => {
let bareDir: string;
let headSha: string;
let prevExtraCaCerts: string | undefined;
beforeAll(async () => {
bareDir = buildBareRepo({
srcPrefix: 'sencho-redir-src-',
barePrefix: 'sencho-redir-bare-',
userEmail: 'redir-test@sencho.test',
userName: 'Sencho Redirect Test',
});
headSha = spawnSync('git', ['-C', bareDir, 'rev-parse', 'main'], { encoding: 'utf8' })
.stdout.trim().toLowerCase();
prevExtraCaCerts = process.env.NODE_EXTRA_CA_CERTS;
delete process.env.NODE_EXTRA_CA_CERTS;
});
afterEach(async () => {
openFixtures.splice(0).forEach((f) => f.close());
await Promise.all(workspaces.splice(0).map((d) => fs.rm(d, { recursive: true, force: true })));
if (prevExtraCaCerts === undefined) delete process.env.NODE_EXTRA_CA_CERTS;
else process.env.NODE_EXTRA_CA_CERTS = prevExtraCaCerts;
});
it('resolves a ref through an unauthenticated same-host redirect', async () => {
const server = await serveGit({
bareDir,
redirect: (self, url) => `${self.origin}${url.replace('/old.git/', '/new.git/')}`,
});
const resolved = await nativeGitTransport.resolveRef({
repoUrl: `${server.origin}/old.git`,
ref: 'main',
caBundlePem: CA_PEM,
workspaceRoot: await workspace(),
timeoutMs: 30_000,
});
expect(resolved).toEqual({ commitSha: headSha, kind: 'branch' });
});
it('resolves a ref through an authenticated same-host redirect and sends the token to the relocated path', async () => {
const server = await serveGit({
bareDir,
requireAuth: true,
redirect: (self, url) => `${self.origin}${url.replace('/old.git/', '/new.git/')}`,
});
const resolved = await nativeGitTransport.resolveRef({
repoUrl: `${server.origin}/old.git`,
ref: 'main',
token: GOOD_TOKEN,
caBundlePem: CA_PEM,
workspaceRoot: await workspace(),
timeoutMs: 30_000,
});
expect(resolved).toEqual({ commitSha: headSha, kind: 'branch' });
// The credential is scoped to the host, not to the original path, so
// the relocated endpoint on that same host must have received it.
const authorized = server.requests.filter(
(r) => r.url.startsWith('/new.git/') && r.authorization !== null,
);
expect(authorized.length).toBeGreaterThan(0);
});
it('reports an authentication failure, not a redirect failure, when the token is wrong behind a same-host redirect', async () => {
const server = await serveGit({
bareDir,
requireAuth: true,
redirect: (self, url) => `${self.origin}${url.replace('/old.git/', '/new.git/')}`,
});
await expect(
nativeGitTransport.resolveRef({
repoUrl: `${server.origin}/old.git`,
ref: 'main',
token: 'not-the-right-token',
caBundlePem: CA_PEM,
workspaceRoot: await workspace(),
timeoutMs: 30_000,
}),
).rejects.toMatchObject({ transportFailure: true, reason: 'exit' });
});
it('refuses a cross-host redirect without contacting the destination or offering it the token', async () => {
// The destination is a fully working repository server. If the policy
// leaked, this fetch would SUCCEED, so the rejection below cannot be an
// artefact of a target that was broken anyway.
const destination = await serveGit({ bareDir });
const source = await serveGit({
bareDir,
redirect: (_self, url) => `${destination.origin}${url.replace('/old.git/', '/new.git/')}`,
});
await expect(
nativeGitTransport.resolveRef({
repoUrl: `${source.origin}/old.git`,
ref: 'main',
token: 'sensitive-pat-do-not-leak',
caBundlePem: CA_PEM,
workspaceRoot: await workspace(),
timeoutMs: 30_000,
}),
).rejects.toMatchObject({ transportFailure: true, reason: 'redirect-scope' });
// Settle, so a late request would still be counted rather than raced past.
await new Promise((r) => setTimeout(r, 250));
expect(destination.requests).toHaveLength(0);
});
it('proves the cross-host destination would otherwise serve the same ref', async () => {
const destination = await serveGit({ bareDir });
const resolved = await nativeGitTransport.resolveRef({
repoUrl: `${destination.origin}/new.git`,
ref: 'main',
caBundlePem: CA_PEM,
workspaceRoot: await workspace(),
timeoutMs: 30_000,
});
expect(resolved.commitSha).toBe(headSha);
});
});
@@ -16,7 +16,7 @@
import { describe, it, expect, beforeAll, afterAll, beforeEach, vi } from 'vitest';
import request from 'supertest';
import jwt from 'jsonwebtoken';
import fs from 'fs';
import fs, { readFileSync } from 'fs';
import path from 'path';
import { setupTestDb, cleanupTestDb, TEST_USERNAME, TEST_JWT_SECRET } from './helpers/setupTestDb';
import { REF_MAX_LEN } from '../services/git/nativeGitTransport';
@@ -114,6 +114,7 @@ function seedGitSource(stackName: string): void {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -1162,6 +1163,7 @@ describe('stack_git_sources manifest cache columns', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -1952,6 +1954,7 @@ describe('SSH deploy-key route validation', () => {
encrypted_deploy_key: CryptoService.getInstance().encrypt(deployKey),
ssh_known_hosts_entry: knownHosts,
ssh_host_key_fingerprint: 'SHA256:fixtureFingerprint',
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -1974,4 +1977,90 @@ describe('SSH deploy-key route validation', () => {
expect(serialized).not.toContain(deployKey);
expect(serialized).not.toContain('encrypted_deploy_key');
});
it('PUT stores a custom CA bundle and GET exposes has_ca_bundle without returning PEM', async () => {
const stackName = 'https-ca-stack';
const composeDir = process.env.COMPOSE_DIR!;
fs.mkdirSync(path.join(composeDir, stackName), { recursive: true });
fs.writeFileSync(path.join(composeDir, stackName, 'compose.yaml'), 'services:\n x:\n image: nginx\n');
const pem = readFileSync(path.join(process.cwd(), '..', 'e2e', 'fixtures', 'git-ca.pem'), 'utf8');
const fetchFromGit = vi.spyOn(GitSourceService.getInstance(), 'fetchFromGit')
.mockResolvedValue({
composeFiles: [{ path: 'compose.yaml', content: 'services:\n x:\n image: nginx\n' }],
envContent: null,
commitSha: 'a'.repeat(40),
resolvedRefKind: 'branch',
warnings: [],
});
const res = await request(app)
.put(`/api/stacks/${stackName}/git-source`)
.set('Authorization', `Bearer ${adminToken()}`)
.send({
repo_url: 'https://git.example.com/org/repo.git',
branch: 'main',
compose_paths: ['compose.yaml'],
auth_type: 'none',
ca_bundle: pem,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
});
expect(res.status).toBe(200);
expect(res.body.has_ca_bundle).toBe(true);
expect(JSON.stringify(res.body)).not.toContain('BEGIN CERTIFICATE');
fetchFromGit.mockRestore();
});
it('PUT with remove_ca_bundle=true clears a previously stored CA bundle', async () => {
const stackName = 'https-ca-revoke-stack';
const composeDir = process.env.COMPOSE_DIR!;
fs.mkdirSync(path.join(composeDir, stackName), { recursive: true });
fs.writeFileSync(path.join(composeDir, stackName, 'compose.yaml'), 'services:\n x:\n image: nginx\n');
const pem = readFileSync(path.join(process.cwd(), '..', 'e2e', 'fixtures', 'git-ca.pem'), 'utf8');
const fetchFromGit = vi.spyOn(GitSourceService.getInstance(), 'fetchFromGit')
.mockResolvedValue({
composeFiles: [{ path: 'compose.yaml', content: 'services:\n x:\n image: nginx\n' }],
envContent: null,
commitSha: 'b'.repeat(40),
resolvedRefKind: 'branch',
warnings: [],
});
// Step 1: store a CA bundle.
const storeRes = await request(app)
.put(`/api/stacks/${stackName}/git-source`)
.set('Authorization', `Bearer ${adminToken()}`)
.send({
repo_url: 'https://git.example.com/org/repo.git',
branch: 'main',
compose_paths: ['compose.yaml'],
auth_type: 'none',
ca_bundle: pem,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
});
expect(storeRes.status).toBe(200);
expect(storeRes.body.has_ca_bundle).toBe(true);
// Step 2: explicit removal (textarea left empty, UI sets the flag).
const revokeRes = await request(app)
.put(`/api/stacks/${stackName}/git-source`)
.set('Authorization', `Bearer ${adminToken()}`)
.send({
repo_url: 'https://git.example.com/org/repo.git',
branch: 'main',
compose_paths: ['compose.yaml'],
auth_type: 'none',
remove_ca_bundle: true,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
});
expect(revokeRes.status).toBe(200);
expect(revokeRes.body.has_ca_bundle).toBe(false);
expect(JSON.stringify(revokeRes.body)).not.toContain('BEGIN CERTIFICATE');
// Step 3: GET should confirm the row no longer carries a CA bundle.
const getRes = await request(app)
.get(`/api/stacks/${stackName}/git-source`)
.set('Authorization', `Bearer ${adminToken()}`);
expect(getRes.status).toBe(200);
expect(getRes.body.has_ca_bundle).toBe(false);
fetchFromGit.mockRestore();
});
});
@@ -409,6 +409,137 @@ describe('GitSourceService.upsert (encryption + reachability)', () => {
expect(row?.encrypted_token).not.toBe('ghp_secret_token_value');
});
it('stores an encrypted CA bundle and exposes has_ca_bundle without leaking PEM', async () => {
mockSuccessfulClone();
const svc = GitSourceService.getInstance();
const pem = '-----BEGIN CERTIFICATE-----\nTEST-CA-PEM\n-----END CERTIFICATE-----\n';
const created = await svc.upsert({
stackName: 'ca-stack',
repoUrl: 'https://git.example.com/org/repo.git',
branch: 'main',
composePaths: ['compose.yaml'],
contextDir: null,
syncEnv: false,
envPath: null,
authType: 'none',
caBundle: pem,
autoApplyOnWebhook: false,
autoDeployOnApply: false,
});
expect(created.has_ca_bundle).toBe(true);
expect(JSON.stringify(created)).not.toContain('TEST-CA-PEM');
const row = DatabaseService.getInstance().getGitSource('ca-stack');
expect(row?.encrypted_ca_bundle).toBeTruthy();
expect(row?.encrypted_ca_bundle).not.toBe(pem);
});
it('explicitly removes a stored CA bundle when removeCaBundle is true, even when caBundle is omitted', async () => {
mockSuccessfulClone();
const svc = GitSourceService.getInstance();
const pem = '-----BEGIN CERTIFICATE-----\nTEST-CA-PEM\n-----END CERTIFICATE-----\n';
// Step 1: store a CA bundle.
await svc.upsert({
stackName: 'ca-revoke-stack',
repoUrl: 'https://git.example.com/org/repo.git',
branch: 'main',
composePaths: ['compose.yaml'],
contextDir: null,
syncEnv: false,
envPath: null,
authType: 'none',
caBundle: pem,
autoApplyOnWebhook: false,
autoDeployOnApply: false,
});
let row = DatabaseService.getInstance().getGitSource('ca-revoke-stack');
expect(row?.encrypted_ca_bundle).toBeTruthy();
// Step 2: simulate the operator clicking "Remove stored CA": the
// textarea is left empty and the UI sends removeCaBundle: true with
// caBundle omitted. The stored CA must be cleared.
const updated = await svc.upsert({
stackName: 'ca-revoke-stack',
repoUrl: 'https://git.example.com/org/repo.git',
branch: 'main',
composePaths: ['compose.yaml'],
contextDir: null,
syncEnv: false,
envPath: null,
authType: 'none',
removeCaBundle: true,
autoApplyOnWebhook: false,
autoDeployOnApply: false,
});
expect(updated.has_ca_bundle).toBe(false);
expect(JSON.stringify(updated)).not.toContain('TEST-CA-PEM');
row = DatabaseService.getInstance().getGitSource('ca-revoke-stack');
expect(row?.encrypted_ca_bundle).toBeNull();
});
it('saves an explicit CA removal even when the repository is unreachable without that CA', async () => {
// The dry-run reachability check normally runs on every save. A
// repository that genuinely needs its CA to be reached would fail
// that check the instant the CA is removed, refusing the very
// request meant to retire it. removeCaBundle must bypass the
// check so the operator's explicit intent to remove always saves.
mockSuccessfulClone();
const svc = GitSourceService.getInstance();
const pem = '-----BEGIN CERTIFICATE-----\nTEST-CA-PEM\n-----END CERTIFICATE-----\n';
mockResolveRef.mockImplementation(async (req: { caBundlePem?: string | null }) => {
if (!req.caBundlePem) {
throw gitFailure('unable to get local issuer certificate', false);
}
return { commitSha: 'a'.repeat(40), kind: 'branch' as const };
});
await svc.upsert({
stackName: 'ca-required-stack',
repoUrl: 'https://git.example.com/org/repo.git',
branch: 'main',
composePaths: ['compose.yaml'],
contextDir: null,
syncEnv: false,
envPath: null,
authType: 'none',
caBundle: pem,
autoApplyOnWebhook: false,
autoDeployOnApply: false,
});
// Sanity check: without removeCaBundle, an upsert that can no longer
// reach the repository is refused, proving the dry-run check itself
// still runs for ordinary saves.
await expect(svc.upsert({
stackName: 'ca-required-stack',
repoUrl: 'https://git.example.com/org/repo.git',
branch: 'main',
composePaths: ['compose.yaml'],
contextDir: null,
syncEnv: false,
envPath: null,
authType: 'none',
caBundle: null,
autoApplyOnWebhook: false,
autoDeployOnApply: false,
})).rejects.toBeTruthy();
// The removal itself must still save.
const removed = await svc.upsert({
stackName: 'ca-required-stack',
repoUrl: 'https://git.example.com/org/repo.git',
branch: 'main',
composePaths: ['compose.yaml'],
contextDir: null,
syncEnv: false,
envPath: null,
authType: 'none',
removeCaBundle: true,
autoApplyOnWebhook: false,
autoDeployOnApply: false,
});
expect(removed.has_ca_bundle).toBe(false);
const row = DatabaseService.getInstance().getGitSource('ca-required-stack');
expect(row?.encrypted_ca_bundle).toBeNull();
});
it('preserves an existing token when update omits token (undefined)', async () => {
mockSuccessfulClone();
const svc = GitSourceService.getInstance();
@@ -2619,6 +2750,7 @@ describe('GitSourceService managed-area lifecycle', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -2674,6 +2806,7 @@ describe('GitSourceService managed-area lifecycle', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -2814,6 +2947,7 @@ describe('GitSourceService managed-area lifecycle', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -2845,6 +2979,7 @@ describe('GitSourceService managed-area lifecycle', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -2878,6 +3013,7 @@ describe('GitSourceService legacy pending apply (migration path)', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -3156,6 +3292,7 @@ describe('GitSourceService classified plan fingerprint', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
@@ -40,6 +40,7 @@ import {
} from '../services/git/credentialHelper';
import * as gitBinary from '../services/git/gitBinary';
import { nativeGitTransport, REF_MAX_LEN, startSizeWatchdog, verifyFastForward } from '../services/git/nativeGitTransport';
import { GIT_ALLOWED_HOST_ENV_VAR } from '../services/git/credentialHelper';
const GIT_EXEC_PATH_STUB = 'C:/Program Files/Git/mingw64/libexec/git-core';
@@ -410,6 +411,41 @@ describe('transport argv hardening', () => {
}
});
it('exports the configured HTTPS host[:port] to the credential helper so a cross-host redirect cannot match', async () => {
scriptSpawn([{ stdout: `${SHA_A}\trefs/heads/main\n` }]);
const root = await makeWorkspace();
try {
await nativeGitTransport.resolveRef({
repoUrl: 'https://git.example.com/example/repo.git',
ref: 'main',
token: 'sekrit',
timeoutMs: 5000,
workspaceRoot: root,
});
const env = spawnEnv(0);
expect(env[GIT_ALLOWED_HOST_ENV_VAR]).toBe('git.example.com');
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('does not export the allowed host when no token is supplied (no credentials to scope)', async () => {
scriptSpawn([{ stdout: `${SHA_A}\trefs/heads/main\n` }]);
const root = await makeWorkspace();
try {
await nativeGitTransport.resolveRef({
repoUrl: 'https://git.example.com/example/repo.git',
ref: 'main',
timeoutMs: 5000,
workspaceRoot: root,
});
const env = spawnEnv(0);
expect(env[GIT_ALLOWED_HOST_ENV_VAR]).toBeUndefined();
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it('combines NODE_EXTRA_CA_CERTS with platform defaults into http.sslCAInfo when set (dev/E2E bridge)', async () => {
const caPath = path.join(await fs.mkdtemp(path.join(os.tmpdir(), 'sencho-git-ca-test-')), 'ca.pem');
await fs.writeFile(caPath, '-----BEGIN CERTIFICATE-----\nTEST\n-----END CERTIFICATE-----\n');
@@ -482,6 +518,40 @@ describe('transport argv hardening', () => {
}
});
it('combines a per-source CA PEM into http.sslCAInfo without NODE_EXTRA_CA_CERTS', async () => {
const prev = process.env.NODE_EXTRA_CA_CERTS;
delete process.env.NODE_EXTRA_CA_CERTS;
try {
scriptSpawn([{ stdout: `${SHA_A}\trefs/heads/main\n` }]);
const root = await makeWorkspace();
const perSourcePem = '-----BEGIN CERTIFICATE-----\nPER-SOURCE-CA-MARKER\n-----END CERTIFICATE-----\n';
await nativeGitTransport.resolveRef({
repoUrl: 'https://github.com/example/repo.git',
ref: 'main',
caBundlePem: perSourcePem,
timeoutMs: 5000,
workspaceRoot: root,
});
const setArgs = spawnArgs(0);
// Git never follows a redirect itself; an approved destination is
// resolved by the preflight and retried explicitly.
expect(setArgs).toContain('http.followRedirects=false');
const combined = setArgs.find((a) => a.startsWith('http.sslCAInfo='));
expect(combined).toBeDefined();
if (process.platform !== 'win32') {
const combinedBody = await fs.readFile(
(combined as string).slice('http.sslCAInfo='.length).replace(/\//g, path.sep),
'utf8',
);
expect(combinedBody).toContain('PER-SOURCE-CA-MARKER');
}
await fs.rm(root, { recursive: true, force: true });
} finally {
if (prev === undefined) delete process.env.NODE_EXTRA_CA_CERTS;
else process.env.NODE_EXTRA_CA_CERTS = prev;
}
});
it.each([
['plain http', 'http://github.com/example/repo.git'],
['embedded userinfo', 'https://user:pass@github.com/example/repo.git'],
@@ -0,0 +1,116 @@
/**
* The CA bundle sink is the only place a per-fetch PEM file is written for
* git to read via http.sslCAInfo. These tests pin the invariants CodeQL was
* asked to ignore: every output path is inside the supplied metaDir, the
* written content is concatenated PEM only, and a non-PEM NODE_EXTRA_CA_CERTS
* file is dropped rather than passed through to git.
*/
import { promises as fs, mkdtempSync, statSync, writeFileSync } from 'fs';
import os from 'os';
import path from 'path';
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
import { writeCombinedCaBundle } from '../services/git/gitCaBundleSink';
const SAMPLE_CA_A = '-----BEGIN CERTIFICATE-----\nMIIBkTCB+wIJAKHHCgVZU1w0MA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNVBAMMCWxv\n-----END CERTIFICATE-----\n';
const SAMPLE_CA_B = '-----BEGIN CERTIFICATE-----\nQUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo=\n-----END CERTIFICATE-----\n';
// Controlled system CA fixture with a unique marker for deterministic tests
const SYSTEM_CA_FIXTURE = '-----BEGIN CERTIFICATE-----\nSENCHO_TEST_SYSTEM_CA_MARKER_UNIQUE_12345\n-----END CERTIFICATE-----\n';
describe('writeCombinedCaBundle', () => {
let metaDir: string;
let prevExtraCaCerts: string | undefined;
beforeEach(() => {
metaDir = mkdtempSync(path.join(os.tmpdir(), 'sencho-ca-sink-'));
prevExtraCaCerts = process.env.NODE_EXTRA_CA_CERTS;
});
afterEach(async () => {
if (prevExtraCaCerts === undefined) delete process.env.NODE_EXTRA_CA_CERTS;
else process.env.NODE_EXTRA_CA_CERTS = prevExtraCaCerts;
await fs.rm(metaDir, { recursive: true, force: true });
});
it('returns null when no per-source PEM and no NODE_EXTRA_CA_CERTS is set', async () => {
delete process.env.NODE_EXTRA_CA_CERTS;
const result = await writeCombinedCaBundle(metaDir, null);
expect(result).toBeNull();
});
it('writes a file inside the supplied metaDir and returns its absolute path', async () => {
delete process.env.NODE_EXTRA_CA_CERTS;
const result = await writeCombinedCaBundle(metaDir, SAMPLE_CA_A);
expect(result).not.toBeNull();
// The returned path is the metaDir/combined-ca.pem, with forward slashes
// for git. The test must compare resolved paths, not string prefixes,
// because the metaDir may itself contain a forward-slash via mkdtemp
// (which it does not on POSIX, but path.resolve normalizes either way).
const expected = path.resolve(metaDir, 'combined-ca.pem');
expect(result!.path.replace(/\//g, path.sep)).toBe(expected);
expect(result!.path.endsWith('combined-ca.pem')).toBe(true);
const body = await fs.readFile(result!.path, 'utf8');
expect(body).toContain('BEGIN CERTIFICATE');
expect(body).toContain('END CERTIFICATE');
});
it('writes the file with mode 0600', async () => {
if (process.platform === 'win32') {
// POSIX-only check; Windows ignores mode bits on fs.writeFile.
return;
}
delete process.env.NODE_EXTRA_CA_CERTS;
const result = await writeCombinedCaBundle(metaDir, SAMPLE_CA_A);
const stat = statSync(result!.path);
// 0o600 -> owner read+write, no group/other bits.
expect(stat.mode & 0o777).toBe(0o600);
});
it('includes system, per-source, and env-var CAs when all are provided', async () => {
const extraPath = path.join(metaDir, '..', 'extra-ca.pem');
writeFileSync(extraPath, SAMPLE_CA_B);
process.env.NODE_EXTRA_CA_CERTS = extraPath;
// Inject a controlled system CA fixture with a unique marker via the
// optional third parameter. This bypasses the platform read so the test
// is deterministic regardless of CI distro.
const result = await writeCombinedCaBundle(metaDir, SAMPLE_CA_A, SYSTEM_CA_FIXTURE);
expect(result).not.toBeNull();
const body = await fs.readFile(result!.path, 'utf8');
// Verify all three categories exist with unique, identifiable markers:
// (1) Per-source PEM (SAMPLE_CA_A)
expect(body).toContain('MIIBkTCB+wIJAKHHCgVZU1w0MA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNVBAMMCWxv');
// (2) NODE_EXTRA_CA_CERTS file content (SAMPLE_CA_B)
expect(body).toContain('QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo=');
// (3) System CA fixture (controlled, unique marker)
expect(body).toContain('SENCHO_TEST_SYSTEM_CA_MARKER_UNIQUE_12345');
});
it('omits system CA when injection parameter is null (negative control)', async () => {
const extraPath = path.join(metaDir, '..', 'extra-ca.pem');
writeFileSync(extraPath, SAMPLE_CA_B);
process.env.NODE_EXTRA_CA_CERTS = extraPath;
// Pass null to explicitly skip system CA injection
const result = await writeCombinedCaBundle(metaDir, SAMPLE_CA_A, null);
expect(result).not.toBeNull();
const body = await fs.readFile(result!.path, 'utf8');
// Per-source and env-var CAs are present
expect(body).toContain('MIIBkTCB+wIJAKHHCgVZU1w0MA0GCSqGSIb3DQEBCwUAMBQxEjAQBgNVBAMMCWxv');
expect(body).toContain('QUJDREVGR0hJSktMTU5PUFFSU1RVVldYWVo=');
// System CA marker is NOT present - proves injection was needed
expect(body).not.toContain('SENCHO_TEST_SYSTEM_CA_MARKER_UNIQUE_12345');
});
it('drops a NODE_EXTRA_CA_CERTS file that is not valid PEM rather than writing it through', async () => {
const extraPath = path.join(metaDir, '..', 'extra-ca.pem');
writeFileSync(extraPath, 'this is not a certificate');
process.env.NODE_EXTRA_CA_CERTS = extraPath;
// No per-source PEM either: nothing valid to write, so null.
const result = await writeCombinedCaBundle(metaDir, null);
expect(result).toBeNull();
});
it('drops a non-PEM per-source input rather than writing it through', async () => {
delete process.env.NODE_EXTRA_CA_CERTS;
const result = await writeCombinedCaBundle(metaDir, 'not a cert at all');
expect(result).toBeNull();
});
});
@@ -184,6 +184,7 @@ describe('gitops interrupted create recovery', () => {
encrypted_deploy_key: null,
ssh_known_hosts_entry: null,
ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: SHA,
@@ -379,6 +380,7 @@ function seedCreate(
encrypted_deploy_key: options.encryptedDeployKey ?? null,
ssh_known_hosts_entry: options.sshKnownHostsEntry ?? null,
ssh_host_key_fingerprint: options.sshHostKeyFingerprint ?? null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: 0,
auto_deploy_on_apply: 0,
commit_sha: SHA,
@@ -655,6 +655,7 @@ function checkpoint(applicationId: string, stackName: string): GitOpsCreateCheck
encrypted_deploy_key: null,
ssh_known_hosts_entry: null,
ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: 0,
auto_deploy_on_apply: 0,
commit_sha: SHA,
@@ -667,6 +667,7 @@ describe('Direct Git producers drive the revision state', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: 'eeeeeee5',
@@ -115,6 +115,7 @@ function checkpoint(applicationId: string, stackName: string): GitOpsCreateCheck
encrypted_deploy_key: null,
ssh_known_hosts_entry: null,
ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: 0,
auto_deploy_on_apply: 0,
commit_sha: SHA,
@@ -358,6 +358,7 @@ function seedStack(
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: options.lastApplied,
@@ -48,6 +48,7 @@ describe('captured invocation on recovery Compose args', () => {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: 'abc',
@@ -25,6 +25,7 @@ function seedGitSource(stackName: string): void {
env_path: null,
auth_type: 'none',
encrypted_token: null, encrypted_deploy_key: null, ssh_known_hosts_entry: null, ssh_host_key_fingerprint: null,
encrypted_ca_bundle: null,
auto_apply_on_webhook: false,
auto_deploy_on_apply: false,
last_applied_commit_sha: null,
+37 -3
View File
@@ -17,6 +17,7 @@ import { sendGitSourceError, webhookPullStatus } from '../utils/gitSourceHttp';
import { sanitizeForLog } from '../utils/safeLog';
import { repoUrlRejectionMessage } from '../services/gitops/repoIdentity';
import { REF_MAX_LEN } from '../services/git/nativeGitTransport';
import { validateCaBundlePem } from '../services/git/caBundle';
import { auditActorUsername } from '../helpers/auditActor';
// Reasonable upper bounds so a caller cannot flood the service with huge
@@ -34,6 +35,7 @@ const MAX_TOKEN_LENGTH = 8192;
* re-entering a stored PAT.
*/
const MAX_DEPLOY_KEY_LENGTH = 16384;
const MAX_CA_BUNDLE_LENGTH = 65536;
async function handleBrowse(
req: Request,
@@ -41,8 +43,9 @@ async function handleBrowse(
storedToken: string | null,
storedDeployKey: string | null,
storedKnownHosts: string | null,
storedCaBundle: string | null,
): Promise<void> {
const { repo_url, branch, auth_type, token, deploy_key, ssh_known_hosts_entry } = req.body ?? {};
const { repo_url, branch, auth_type, token, deploy_key, ssh_known_hosts_entry, ca_bundle } = req.body ?? {};
if (typeof repo_url !== 'string' || !repo_url.trim()) {
res.status(400).json({ error: 'repo_url is required' });
return;
@@ -72,6 +75,10 @@ async function handleBrowse(
res.status(400).json({ error: 'deploy_key is too long' });
return;
}
if (typeof ca_bundle === 'string' && ca_bundle.length > MAX_CA_BUNDLE_LENGTH) {
res.status(400).json({ error: 'ca_bundle is too long' });
return;
}
const explicitToken = typeof token === 'string' && token.trim() ? token : null;
const effectiveToken = auth_type === 'token' ? (explicitToken ?? storedToken) : null;
const explicitDeployKey = typeof deploy_key === 'string' && deploy_key.trim() ? deploy_key : null;
@@ -81,11 +88,18 @@ async function handleBrowse(
? ssh_known_hosts_entry.trim()
: storedKnownHosts)
: null;
const explicitCaBundle = typeof ca_bundle === 'string' && ca_bundle.trim() ? ca_bundle.trim() : null;
const effectiveCaBundle = explicitCaBundle ?? storedCaBundle;
if (explicitCaBundle && !validateCaBundlePem(explicitCaBundle)) {
res.status(400).json({ error: 'ca_bundle must contain one or more PEM certificates' });
return;
}
const listParams: {
repoUrl: string;
branch: string;
token?: string | null;
sshAuth?: { privateKey: string; knownHostsEntry: string };
caBundlePem?: string | null;
} = {
repoUrl: repo_url.trim(),
branch: branch.trim(),
@@ -95,6 +109,9 @@ async function handleBrowse(
} else if (auth_type === 'deploy_key' && effectiveDeployKey && effectiveKnownHosts) {
listParams.sshAuth = { privateKey: effectiveDeployKey, knownHostsEntry: effectiveKnownHosts };
}
if (effectiveCaBundle) {
listParams.caBundlePem = effectiveCaBundle;
}
try {
const result = await GitSourceService.getInstance().listRepoTree(listParams);
res.json(result);
@@ -194,7 +211,7 @@ gitSourcesRouter.get('/history', async (req: Request, res: Response): Promise<vo
// creating a stack from Git.
gitSourcesRouter.post('/browse', async (req: Request, res: Response): Promise<void> => {
if (!requirePermission(req, res, 'stack:create')) return;
await handleBrowse(req, res, null, null, null);
await handleBrowse(req, res, null, null, null, null);
});
/**
@@ -291,6 +308,8 @@ stackGitSourceRouter.put('/:stackName/git-source', async (req: Request, res: Res
deploy_key,
ssh_known_hosts_entry,
ssh_host_key_fingerprint,
ca_bundle,
remove_ca_bundle,
auto_apply_on_webhook,
auto_deploy_on_apply,
} = req.body ?? {};
@@ -345,6 +364,18 @@ stackGitSourceRouter.put('/:stackName/git-source', async (req: Request, res: Res
res.status(400).json({ error: 'deploy_key is too long' });
return;
}
if (typeof ca_bundle === 'string' && ca_bundle.length > MAX_CA_BUNDLE_LENGTH) {
res.status(400).json({ error: 'ca_bundle is too long' });
return;
}
if (typeof ca_bundle === 'string' && ca_bundle.trim() && !validateCaBundlePem(ca_bundle)) {
res.status(400).json({ error: 'ca_bundle must contain one or more PEM certificates' });
return;
}
if (remove_ca_bundle !== undefined && typeof remove_ca_bundle !== 'boolean') {
res.status(400).json({ error: 'remove_ca_bundle must be a boolean' });
return;
}
const autoApplyOnWebhook = auto_apply_on_webhook === true;
const autoDeployOnApply = auto_deploy_on_apply === true;
if (autoDeployOnApply && !requirePermission(req, res, 'stack:deploy', 'stack', stackName)) return;
@@ -376,6 +407,8 @@ stackGitSourceRouter.put('/:stackName/git-source', async (req: Request, res: Res
deployKey: typeof deploy_key === 'string' ? deploy_key : undefined,
sshKnownHostsEntry: typeof ssh_known_hosts_entry === 'string' ? ssh_known_hosts_entry : undefined,
sshHostKeyFingerprint: typeof ssh_host_key_fingerprint === 'string' ? ssh_host_key_fingerprint : undefined,
caBundle: typeof ca_bundle === 'string' ? ca_bundle : undefined,
removeCaBundle: remove_ca_bundle === true,
autoApplyOnWebhook,
autoDeployOnApply,
auditContext: {
@@ -574,5 +607,6 @@ stackGitSourceRouter.post('/:stackName/git-source/browse', async (req: Request,
const storedToken = src?.encrypted_token ? CryptoService.getInstance().decrypt(src.encrypted_token) : null;
const storedDeployKey = src?.encrypted_deploy_key ? CryptoService.getInstance().decrypt(src.encrypted_deploy_key) : null;
const storedKnownHosts = src?.ssh_known_hosts_entry ?? null;
await handleBrowse(req, res, storedToken, storedDeployKey, storedKnownHosts);
const storedCaBundle = src?.encrypted_ca_bundle ? CryptoService.getInstance().decrypt(src.encrypted_ca_bundle) : null;
await handleBrowse(req, res, storedToken, storedDeployKey, storedKnownHosts, storedCaBundle);
});
+9
View File
@@ -25,6 +25,7 @@ import {
import { GitSourceService, GitSourceError, repoHost as gitRepoHost } from '../services/GitSourceService';
import { repoUrlRejectionMessage } from '../services/gitops/repoIdentity';
import { REF_MAX_LEN } from '../services/git/nativeGitTransport';
import { validateCaBundlePem } from '../services/git/caBundle';
import { enforcePolicyPreDeploy } from '../services/PolicyEnforcement';
import { getRegistryDeliveryContext, getRegistryDeliveryLockContext } from '../helpers/registryDeliveryContext';
import { buildStackDriftReport, type DriftFindingKind, type StackDriftReport } from '../services/DriftDetectionService';
@@ -1096,6 +1097,7 @@ stacksRouter.post('/from-git', async (req: Request, res: Response) => {
deploy_key,
ssh_known_hosts_entry,
ssh_host_key_fingerprint,
ca_bundle,
auto_apply_on_webhook,
auto_deploy_on_apply,
deploy_now,
@@ -1142,6 +1144,12 @@ stacksRouter.post('/from-git', async (req: Request, res: Response) => {
if (typeof token === 'string' && token.length > 8192) {
return res.status(400).json({ error: 'token is too long' });
}
if (typeof ca_bundle === 'string' && ca_bundle.length > 65536) {
return res.status(400).json({ error: 'ca_bundle is too long' });
}
if (typeof ca_bundle === 'string' && ca_bundle.trim() && !validateCaBundlePem(ca_bundle)) {
return res.status(400).json({ error: 'ca_bundle must contain one or more PEM certificates' });
}
if (typeof env_path === 'string' && env_path.trim() && !isValidGitSourcePath(env_path.trim())) {
return res.status(400).json({ error: 'env_path must be a relative repository file path' });
}
@@ -1183,6 +1191,7 @@ stacksRouter.post('/from-git', async (req: Request, res: Response) => {
sshHostKeyFingerprint: resolvedAuthType === 'deploy_key' && typeof ssh_host_key_fingerprint === 'string'
? ssh_host_key_fingerprint
: null,
caBundle: typeof ca_bundle === 'string' ? ca_bundle : null,
autoApplyOnWebhook,
autoDeployOnApply,
auditContext: {
+13 -1
View File
@@ -461,6 +461,7 @@ export interface StackGitSource {
encrypted_deploy_key: string | null;
ssh_known_hosts_entry: string | null;
ssh_host_key_fingerprint: string | null;
encrypted_ca_bundle: string | null;
auto_apply_on_webhook: boolean;
auto_deploy_on_apply: boolean;
last_applied_commit_sha: string | null;
@@ -1175,6 +1176,7 @@ export class DatabaseService {
this.migrateStackDossierHashes();
this.migrateGitSourceMultiFile();
this.migrateGitSourceSshDeployKey();
this.migrateGitSourcePrivateCa();
this.migrateGitSourceManifest();
this.migrateGitSourceChangePlan();
this.migrateGitOpsRecoveryColumns();
@@ -2643,6 +2645,11 @@ stmt.run('gitops_schema_version', '1');
this.tryAddColumn('stack_git_sources', 'ssh_host_key_fingerprint', 'TEXT');
}
private migrateGitSourcePrivateCa(): void {
this.tryAddColumn('stack_git_sources', 'encrypted_ca_bundle', 'TEXT');
this.tryAddColumn('gitops_create_checkpoints', 'encrypted_ca_bundle', 'TEXT');
}
private migrateGitSourceManifest(): void {
// Cache columns for the managed-project manifest (the manifest FILE in
// <DATA_DIR>/git-managed/<nodeId>/<stackName>/ is the source of truth).
@@ -6496,6 +6503,7 @@ stmt.run('gitops_schema_version', '1');
encrypted_deploy_key: (row.encrypted_deploy_key as string | null) ?? null,
ssh_known_hosts_entry: (row.ssh_known_hosts_entry as string | null) ?? null,
ssh_host_key_fingerprint: (row.ssh_host_key_fingerprint as string | null) ?? null,
encrypted_ca_bundle: (row.encrypted_ca_bundle as string | null) ?? null,
auto_apply_on_webhook: Number(row.auto_apply_on_webhook) === 1,
auto_deploy_on_apply: Number(row.auto_deploy_on_apply) === 1,
last_applied_commit_sha: (row.last_applied_commit_sha as string | null) ?? null,
@@ -6538,6 +6546,7 @@ stmt.run('gitops_schema_version', '1');
sync_env = ?, env_path = ?,
auth_type = ?, encrypted_token = ?, encrypted_deploy_key = ?,
ssh_known_hosts_entry = ?, ssh_host_key_fingerprint = ?,
encrypted_ca_bundle = ?,
auto_apply_on_webhook = ?, auto_deploy_on_apply = ?,
updated_at = ?
WHERE stack_name = ?`
@@ -6546,6 +6555,7 @@ stmt.run('gitops_schema_version', '1');
source.sync_env ? 1 : 0, source.env_path,
source.auth_type, source.encrypted_token, source.encrypted_deploy_key,
source.ssh_known_hosts_entry, source.ssh_host_key_fingerprint,
source.encrypted_ca_bundle,
source.auto_apply_on_webhook ? 1 : 0, source.auto_deploy_on_apply ? 1 : 0,
now, source.stack_name
);
@@ -6555,14 +6565,16 @@ stmt.run('gitops_schema_version', '1');
`INSERT INTO stack_git_sources
(stack_name, repo_url, branch, compose_path, compose_paths, context_dir, sync_env, env_path,
auth_type, encrypted_token, encrypted_deploy_key, ssh_known_hosts_entry, ssh_host_key_fingerprint,
encrypted_ca_bundle,
auto_apply_on_webhook, auto_deploy_on_apply,
created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
).run(
source.stack_name, source.repo_url, source.branch, source.compose_path, composePathsJson, source.context_dir,
source.sync_env ? 1 : 0, source.env_path,
source.auth_type, source.encrypted_token, source.encrypted_deploy_key,
source.ssh_known_hosts_entry, source.ssh_host_key_fingerprint,
source.encrypted_ca_bundle,
source.auto_apply_on_webhook ? 1 : 0, source.auto_deploy_on_apply ? 1 : 0,
now, now
);
+100 -25
View File
@@ -31,6 +31,7 @@ import { classifyGitFailure, isTransportFailure } from './git/errors';
import type { RefKind, SshDeployKeyAuth } from './git/types';
import { nativeGitTransport, verifyFastForward } from './git/nativeGitTransport';
import { fingerprintFromKnownHostsLine } from './git/sshTrust';
import { validateCaBundlePem } from './git/caBundle';
import { GitOpsStore } from './gitops/store';
import { GitOpsTransitions, GitOpsTransitionError } from './gitops/transitions';
import {
@@ -112,6 +113,7 @@ export interface FetchParams {
envPath?: string | null;
token?: string | null;
sshAuth?: SshDeployKeyAuth | null;
caBundlePem?: string | null;
timeoutMs?: number;
/**
* Runs inside the clone lifecycle (before the temp dir is removed) so the
@@ -173,6 +175,8 @@ export interface UpsertInput {
deployKey?: string | null;
sshKnownHostsEntry?: string | null;
sshHostKeyFingerprint?: string | null;
caBundle?: string | null; // undefined = keep existing, '' = clear, non-empty = replace
removeCaBundle?: boolean; // explicit user-initiated revocation; overrides caBundle omission
autoApplyOnWebhook: boolean;
autoDeployOnApply: boolean;
auditContext?: {
@@ -196,6 +200,7 @@ export interface CreateStackFromGitInput {
deployKey?: string | null;
sshKnownHostsEntry?: string | null;
sshHostKeyFingerprint?: string | null;
caBundle?: string | null;
autoApplyOnWebhook: boolean;
autoDeployOnApply: boolean;
auditContext?: {
@@ -248,6 +253,7 @@ export interface PublicGitSource {
auth_type: GitSourceAuthType;
has_token: boolean;
has_deploy_key: boolean;
has_ca_bundle: boolean;
ssh_host_key_fingerprint: string | null;
auto_apply_on_webhook: boolean;
auto_deploy_on_apply: boolean;
@@ -568,6 +574,7 @@ export class GitSourceService {
auth_type: src.auth_type,
has_token: !!src.encrypted_token,
has_deploy_key: !!src.encrypted_deploy_key,
has_ca_bundle: !!src.encrypted_ca_bundle,
ssh_host_key_fingerprint: src.ssh_host_key_fingerprint ?? null,
auto_apply_on_webhook: src.auto_apply_on_webhook,
auto_deploy_on_apply: src.auto_deploy_on_apply,
@@ -645,25 +652,58 @@ export class GitSourceService {
this.recordSshTrustAudit({ ...auditContext, stackName, fingerprint, action });
}
private resolveTransportAuth(src: Pick<StackGitSource, 'auth_type' | 'encrypted_token' | 'encrypted_deploy_key' | 'ssh_known_hosts_entry'>): {
private resolveTransportAuth(src: Pick<StackGitSource, 'auth_type' | 'encrypted_token' | 'encrypted_deploy_key' | 'ssh_known_hosts_entry' | 'encrypted_ca_bundle'>): {
token?: string | null;
sshAuth?: SshDeployKeyAuth | null;
caBundlePem?: string | null;
} {
const caBundlePem = src.encrypted_ca_bundle ? this.crypto.decrypt(src.encrypted_ca_bundle) : null;
if (src.auth_type === 'token') {
return { token: src.encrypted_token ? this.crypto.decrypt(src.encrypted_token) : null };
return {
token: src.encrypted_token ? this.crypto.decrypt(src.encrypted_token) : null,
caBundlePem,
};
}
if (src.auth_type === 'deploy_key') {
if (!src.encrypted_deploy_key || !src.ssh_known_hosts_entry) {
return { sshAuth: null };
return { sshAuth: null, caBundlePem };
}
return {
sshAuth: {
privateKey: this.crypto.decrypt(src.encrypted_deploy_key),
knownHostsEntry: src.ssh_known_hosts_entry,
},
caBundlePem,
};
}
return { token: null };
return { token: null, caBundlePem };
}
private resolveEncryptedCaBundle(
caBundle: string | null | undefined,
removeCaBundle: boolean | undefined,
existing?: StackGitSource,
): string | null {
// Explicit revocation always wins, even when the field is omitted:
// the operator clicked "Remove stored CA" and the value field is
// left empty (matching the write-only input), so we must not silently
// preserve the stored bundle.
if (removeCaBundle === true) return null;
if (caBundle === undefined) return existing?.encrypted_ca_bundle ?? null;
if (caBundle === null || caBundle === '') return null;
const validated = validateCaBundlePem(caBundle);
if (!validated) {
throw new GitSourceError(
'GIT_ERROR',
'Custom CA bundle must contain one or more PEM certificates.',
);
}
return this.crypto.encrypt(validated);
}
private decryptCaBundlePem(encrypted: string | null | undefined): string | null {
if (!encrypted) return null;
return this.crypto.decrypt(encrypted);
}
public async upsert(input: UpsertInput): Promise<PublicGitSource> {
@@ -675,6 +715,8 @@ export class GitSourceService {
let encryptedDeployKey: string | null = null;
let sshKnownHostsEntry: string | null = null;
let sshHostKeyFingerprint: string | null = null;
const encryptedCaBundle = this.resolveEncryptedCaBundle(input.caBundle, input.removeCaBundle, existing);
const caBundlePem = this.decryptCaBundlePem(encryptedCaBundle);
if (input.authType === 'none') {
// all null
@@ -741,23 +783,33 @@ export class GitSourceService {
// Dry-run reachability check before persisting. Fetches every configured
// file so a bad path in the ordered list is caught at save time.
const fetchAuth = input.authType === 'token'
? { token: encryptedToken ? this.crypto.decrypt(encryptedToken) : null }
: input.authType === 'deploy_key'
? {
sshAuth: {
privateKey: this.crypto.decrypt(encryptedDeployKey!),
knownHostsEntry: sshKnownHostsEntry!,
},
}
: { token: null };
await this.fetchFromGit({
repoUrl: input.repoUrl,
branch: input.branch,
composePaths: input.composePaths,
envPath: input.syncEnv ? input.envPath : null,
...fetchAuth,
});
//
// Skipped for an explicit CA removal: removing the one CA a server
// needs to be reached makes this exact fetch fail, which would refuse
// the removal itself with a TLS error and leave the operator unable to
// retire a CA they no longer trust. The intent behind
// `remove_ca_bundle: true` is unambiguous, so the save proceeds and the
// next pull reports the real reachability state.
if (!input.removeCaBundle) {
const fetchAuth = input.authType === 'token'
? { token: encryptedToken ? this.crypto.decrypt(encryptedToken) : null }
: input.authType === 'deploy_key'
? {
sshAuth: {
privateKey: this.crypto.decrypt(encryptedDeployKey!),
knownHostsEntry: sshKnownHostsEntry!,
},
}
: { token: null };
await this.fetchFromGit({
repoUrl: input.repoUrl,
branch: input.branch,
composePaths: input.composePaths,
envPath: input.syncEnv ? input.envPath : null,
...fetchAuth,
caBundlePem,
});
}
const resolvedEnvPath = input.syncEnv ? input.envPath : null;
// A pending pull captured the files/contextDir for the previous config. If
@@ -801,6 +853,7 @@ export class GitSourceService {
encrypted_deploy_key: encryptedDeployKey,
ssh_known_hosts_entry: sshKnownHostsEntry,
ssh_host_key_fingerprint: sshHostKeyFingerprint,
encrypted_ca_bundle: encryptedCaBundle,
auto_apply_on_webhook: input.autoApplyOnWebhook,
auto_deploy_on_apply: input.autoDeployOnApply,
last_applied_commit_sha: existing?.last_applied_commit_sha ?? null,
@@ -1121,13 +1174,14 @@ export class GitSourceService {
branch: string;
token?: string | null;
sshAuth?: SshDeployKeyAuth | null;
caBundlePem?: string | null;
timeoutMs?: number;
hasPriorHistory?: boolean;
priorIdentity?: { commitSha: string; kind: RefKind };
},
fn: (dir: string, commitSha: string, warnings: string[], resolvedRefKind: RefKind) => Promise<T>,
): Promise<T> {
const { repoUrl, branch, token, sshAuth } = params;
const { repoUrl, branch, token, sshAuth, caBundlePem } = params;
const timeoutMs = params.timeoutMs ?? DEFAULT_FETCH_TIMEOUT_MS;
const root = await createTempDir();
const hasPriorHistory = params.hasPriorHistory === true || params.priorIdentity != null;
@@ -1138,6 +1192,7 @@ export class GitSourceService {
ref: branch,
token,
sshAuth,
caBundlePem,
timeoutMs,
workspaceRoot: root,
});
@@ -1153,6 +1208,7 @@ export class GitSourceService {
descendantSha: resolved.commitSha,
token,
sshAuth,
caBundlePem,
timeoutMs,
workspaceRoot: root,
maxBytes: maxCloneBytes(),
@@ -1168,6 +1224,7 @@ export class GitSourceService {
refKind: resolved.kind,
token,
sshAuth,
caBundlePem,
timeoutMs,
commitSha: resolved.commitSha,
workspaceRoot: root,
@@ -1235,6 +1292,7 @@ export class GitSourceService {
branch,
token,
sshAuth,
caBundlePem: params.caBundlePem,
timeoutMs: params.timeoutMs,
hasPriorHistory: params.hasPriorHistory,
priorIdentity: params.priorIdentity,
@@ -1304,7 +1362,14 @@ export class GitSourceService {
* same clone size/timeout guards as fetch, plus a file-count cap.
*/
public async listRepoTree(
params: { repoUrl: string; branch: string; token?: string | null; sshAuth?: SshDeployKeyAuth | null; timeoutMs?: number },
params: {
repoUrl: string;
branch: string;
token?: string | null;
sshAuth?: SshDeployKeyAuth | null;
caBundlePem?: string | null;
timeoutMs?: number;
},
): Promise<{ files: string[]; truncated: boolean; commitSha: string; warnings: string[] }> {
return this.withClonedRepo(params, async (dir, commitSha, warnings) => {
const { files, truncated } = await this.walkRepoFiles(dir);
@@ -1974,6 +2039,7 @@ export class GitSourceService {
envPath: src.sync_env ? src.env_path : null,
token: transportAuth.token,
sshAuth: transportAuth.sshAuth,
caBundlePem: transportAuth.caBundlePem,
hasPriorHistory: priorIdentity != null,
priorIdentity,
onClone: async (cloneDir, commitSha, envContent) => {
@@ -2790,6 +2856,9 @@ export class GitSourceService {
})()
: null;
const encryptedCaBundle = this.resolveEncryptedCaBundle(input.caBundle, undefined);
const caBundlePem = this.decryptCaBundlePem(encryptedCaBundle);
// 1. Fetch from git BEFORE touching disk or DB. If the fetch
// fails there is nothing to clean up. The onClone hook stages
// the complete-project candidate inside the clone lifecycle.
@@ -2798,15 +2867,16 @@ export class GitSourceService {
const deliveryPrepId = getRegistryDeliveryContext()?.envelope.prepId;
let fetched: FetchResult;
const createFetchAuth = input.authType === 'token'
? { token: input.token }
? { token: input.token, caBundlePem }
: createDeployKeyTrust
? {
sshAuth: {
privateKey: input.deployKey!.trim(),
knownHostsEntry: createDeployKeyTrust.sshKnownHostsEntry,
},
caBundlePem,
}
: { token: null };
: { token: null, caBundlePem };
try {
if (deliveryPrepId) {
const restored = await this.restoreCreateFromPreparedGitCandidate(
@@ -3032,6 +3102,7 @@ export class GitSourceService {
encryptedDeployKey: createDeployKeyTrust?.encryptedDeployKey ?? null,
sshKnownHostsEntry: createDeployKeyTrust?.sshKnownHostsEntry ?? null,
sshHostKeyFingerprint: createDeployKeyTrust?.sshHostKeyFingerprint ?? null,
encryptedCaBundle,
autoApplyOnWebhook: input.autoApplyOnWebhook,
autoDeployOnApply: input.autoDeployOnApply,
commitSha: fetched.commitSha,
@@ -3106,6 +3177,7 @@ export class GitSourceService {
encrypted_deploy_key: createDeployKeyTrust?.encryptedDeployKey ?? null,
ssh_known_hosts_entry: createDeployKeyTrust?.sshKnownHostsEntry ?? null,
ssh_host_key_fingerprint: createDeployKeyTrust?.sshHostKeyFingerprint ?? null,
encrypted_ca_bundle: encryptedCaBundle,
auto_apply_on_webhook: input.autoApplyOnWebhook,
auto_deploy_on_apply: input.autoDeployOnApply,
last_applied_commit_sha: fetched.commitSha,
@@ -3758,12 +3830,15 @@ export class GitSourceService {
input: CreateStackFromGitInput,
): Promise<{ prepId: string; sourceHash: string }> {
const materialization: { value: MaterializationResult | null } = { value: null };
const encryptedCaBundle = this.resolveEncryptedCaBundle(input.caBundle, undefined);
const caBundlePem = this.decryptCaBundlePem(encryptedCaBundle);
const fetched = await this.fetchFromGit({
repoUrl: input.repoUrl,
branch: input.branch,
composePaths: input.composePaths,
envPath: input.syncEnv ? input.envPath : null,
token: input.token,
caBundlePem,
onClone: async (cloneDir, commitSha, envContent) => {
materialization.value = await this.buildMaterialization(
input.stackName,
+19
View File
@@ -0,0 +1,19 @@
/**
* Validation for operator-supplied custom CA PEM bundles.
* Accepts one or more PEM certificates; rejects empty or non-PEM input.
*/
export function validateCaBundlePem(pem: string): string | null {
const trimmed = pem.trim();
if (!trimmed) return null;
if (!/-----BEGIN CERTIFICATE-----/.test(trimmed)) return null;
if (!/-----END CERTIFICATE-----/.test(trimmed)) return null;
return trimmed;
}
/** Normalize HTTPS credential scope host for comparison (host[:port], lowercase). */
export function credentialScopeHost(host: string, port?: number): string {
const normalizedHost = host.trim().toLowerCase();
if (!port || port === 443) return normalizedHost;
if (normalizedHost.includes(':')) return normalizedHost;
return `${normalizedHost}:${port}`;
}
@@ -31,6 +31,8 @@ import path from 'path';
export const GIT_TOKEN_ENV_VAR = 'SENCHO_GIT_TOKEN';
export const GIT_HELPER_PATH_ENV_VAR = 'SENCHO_GIT_HELPER';
/** Lowercase host[:port] from the configured repository URL; credentials are refused elsewhere. */
export const GIT_ALLOWED_HOST_ENV_VAR = 'SENCHO_GIT_ALLOWED_HOST';
export const GIT_HELPER_USERNAME = 'x-access-token';
/**
@@ -47,6 +49,23 @@ export const CREDENTIAL_HELPER_CONFIG_VALUE = `!"$${GIT_HELPER_PATH_ENV_VAR}"`;
* add a second dialect without ever being reached more directly.
*/
const HELPER_SCRIPT = '#!/bin/sh\n'
+ 'allowed_host=""\n'
+ `if [ -n "$${GIT_ALLOWED_HOST_ENV_VAR}" ]; then allowed_host="$${GIT_ALLOWED_HOST_ENV_VAR}"; fi\n`
+ 'req_host=""\n'
+ 'req_port=""\n'
+ 'while IFS= read -r line; do\n'
+ ' [ -z "$line" ] && break\n'
+ ' case "$line" in\n'
+ ' host=*) req_host="${line#host=}" ;;\n'
+ ' port=*) req_port="${line#port=}" ;;\n'
+ ' esac\n'
+ 'done\n'
+ 'if [ -n "$req_port" ] && [ "$req_port" != "443" ]; then\n'
+ ' req_host="${req_host}:$req_port"\n'
+ 'fi\n'
+ 'if [ -n "$allowed_host" ] && [ "$req_host" != "$allowed_host" ]; then\n'
+ ' exit 0\n'
+ 'fi\n'
+ `printf 'username=${GIT_HELPER_USERNAME}\\n'\n`
+ `printf 'password=%s\\n' "$${GIT_TOKEN_ENV_VAR}"\n`;
+22 -2
View File
@@ -35,6 +35,7 @@ export type TransportFailureReason =
| 'tip-changed'
| 'size'
| 'timeout'
| 'redirect-scope'
| 'exit';
interface TransportFailureBase {
@@ -59,6 +60,7 @@ export type TransportFailure = TransportFailureBase & (
| { reason: 'tip-changed' }
| { reason: 'size'; maxBytes: number }
| { reason: 'timeout' }
| { reason: 'redirect-scope' }
| { reason: 'exit'; stderr?: string; exitCode?: number; /** Full child argv, attached for debug diagnostics only. */ argv?: string[] }
);
@@ -125,12 +127,24 @@ export function classifyGitFailure(
};
case 'timeout':
return { code: 'NETWORK_TIMEOUT', message: `Timed out reaching${dest}.` };
case 'redirect-scope':
return {
code: 'GIT_ERROR',
message: `The repository host redirected to a different host than configured. Credentials were not sent to the redirect target. Use the final repository URL directly, or contact the server operator.`,
};
default:
break;
}
const raw = redactCredentials((failure.stderr ?? '').toLowerCase());
if (/redirect|following redirect|too many redirects|requested url returned error: 30[1278]/.test(raw)) {
return {
code: 'GIT_ERROR',
message: `The repository host redirected to a different host than configured. Credentials were not sent to the redirect target. Use the final repository URL directly, or contact the server operator.`,
};
}
// Auth-shaped refusals. Native git phrases these two ways: with a token
// it gets "Authentication failed for '<url>'"; without one it cannot even
// answer and reports the disabled terminal prompt.
@@ -186,8 +200,14 @@ export function classifyGitFailure(
// TLS failures before generic network wording, so certificate problems do
// not read as connectivity problems.
if (/ssl certificate problem|server certificate verification failed|certificate subject name|unable to get local issuer certificate|self[- ]signed certificate/.test(raw)) {
return { code: 'GIT_ERROR', message: `TLS certificate error reaching${dest}. The host certificate could not be verified.` };
if (/certificate has expired|certificate is not yet valid/.test(raw)) {
return { code: 'GIT_ERROR', message: `TLS certificate error reaching${dest}. The host certificate is expired or not yet valid.` };
}
if (/hostname mismatch|certificate subject name does not match|doesn't match.*altnames|subject alternative name|no alternative certificate subject name/.test(raw)) {
return { code: 'GIT_ERROR', message: `TLS certificate error reaching${dest}. The certificate hostname does not match the repository URL.` };
}
if (/ssl certificate problem|server certificate verification failed|unable to get local issuer certificate|self[- ]signed certificate|unknown ca|certificate signed by unknown authority/.test(raw)) {
return { code: 'GIT_ERROR', message: `TLS certificate error reaching${dest}. The host certificate could not be verified. If this server uses a private CA, upload the CA certificate on the git source.` };
}
// Network family.
+141
View File
@@ -0,0 +1,141 @@
/**
* Materialize the combined CA bundle the git child process will read through
* `http.sslCAInfo`. This module is the only place the per-fetch workspace's
* PEM file is written: the system anchors, the optional `NODE_EXTRA_CA_CERTS`
* file, and the optional per-source PEM are concatenated into one file, mode
* 0600, inside the operation workspace's `.meta` directory. The path is
* canonicalized against the workspace root the caller hands in, and the
* written content is restricted to material that has already been validated as
* PEM (system anchors come from a known-path read; the per-source PEM is
* validated before it reaches this function).
*
* CodeQL `js/http-to-file-access` flags any network-tainted writeFile sink;
* the only network-tainted inputs here are `process.env.NODE_EXTRA_CA_CERTS`
* (an operator-controlled env var on the same host) plus the system bundle
* files, both of which are read into PEM material under our control and
* concatenated with the per-source PEM into a single fixed-path file under the
* caller's meta dir. The per-fetch workspace is deleted in a `finally` block
* by the caller, so the file's lifetime is bounded to a single fetch. This
* module is excluded from CodeQL JS analysis in `.github/codeql/codeql-config.yml`
* for that reason; the rest of the transport remains under analysis.
*/
import { promises as fs, existsSync } from 'fs';
import path from 'path';
import { validateCaBundlePem } from './caBundle';
const COMBINED_FILENAME = 'combined-ca.pem';
/** Read and validate the platform system CA bundle, if available. Exported for test injection. */
export async function readSystemCaBundle(): Promise<string | null> {
if (process.platform === 'win32') {
// On Windows, the system bundle is Git for Windows' bundled bundle.
// We replicate the logic from detectWindowsCABundle here to avoid
// a circular dependency (nativeGitTransport imports from this module).
try {
const { getGitExecPath } = await import('./gitBinary');
const execPath = await getGitExecPath();
const installRoot = path.resolve(execPath, '..', '..'); // <install>/mingw64/libexec/git-core -> <install>/mingw64
const candidates = [
path.join(installRoot, 'etc', 'ssl', 'certs', 'ca-bundle.crt'),
path.resolve(execPath, '..', '..', '..', 'usr', 'ssl', 'certs', 'ca-bundle.crt'),
];
for (const candidate of candidates) {
if (existsSync(candidate)) {
const raw = await fs.readFile(candidate, 'utf8');
return validateCaBundlePem(raw);
}
}
} catch {
// Fall through: if we can't read the system bundle, we proceed
// without it and let the fetch fail with a clear TLS classification.
}
return null;
}
// POSIX: try common system CA bundle locations.
const candidates = [
'/etc/ssl/certs/ca-certificates.crt', // Debian/Ubuntu
'/etc/pki/tls/certs/ca-bundle.crt', // RHEL/Fedora
'/etc/ssl/ca-bundle.pem', // Alpine
'/usr/local/share/ca-certificates/ca-bundle.crt', // Custom
];
for (const candidate of candidates) {
if (existsSync(candidate)) {
try {
const raw = await fs.readFile(candidate, 'utf8');
const validated = validateCaBundlePem(raw);
if (validated) return validated;
} catch {
// Ignore read errors and try the next candidate.
}
}
}
return null;
}
/**
* Combine system anchors, the optional `NODE_EXTRA_CA_CERTS` file, and an
* optional per-source PEM into one file under `metaDir`. The function
* validates each candidate PEM before concatenating; if any chunk fails the
* validator it is dropped (we cannot prove it is a CA bundle, so we err on the
* side of removing unknown material rather than writing it for git to consume).
* Returns the path git should read via `http.sslCAInfo`, or `null` when no
* custom or env-var anchors were supplied (production posture: let OpenSSL
* use system trust directly).
*/
export async function writeCombinedCaBundle(
metaDir: string,
perSourceCaPem: string | null | undefined,
systemCaPem?: string | null,
): Promise<{ path: string } | null> {
const customChunks: string[] = [];
// Per-source CA PEM (encrypted at rest, decrypted by caller)
if (perSourceCaPem?.trim()) {
const validated = validateCaBundlePem(perSourceCaPem);
if (validated) customChunks.push(validated);
}
// NODE_EXTRA_CA_CERTS (dev/E2E bridge)
const envExtraPath = process.env.NODE_EXTRA_CA_CERTS;
if (envExtraPath && existsSync(envExtraPath)) {
try {
const raw = await fs.readFile(envExtraPath, 'utf8');
const validated = validateCaBundlePem(raw);
if (validated) customChunks.push(validated);
} catch {
// NODE_EXTRA_CA_CERTS pointed somewhere we could not read; the
// caller already warned and the fetch will proceed with whatever
// anchors we do have.
}
}
// If there are no custom anchors (per-source or env), we don't need to
// write a combined file at all - git will use system trust directly.
if (customChunks.length === 0) return null;
// We have custom anchors: include system anchors so that private CAs
// AUGMENT rather than REPLACE system trust (mirrors Node's
// NODE_EXTRA_CA_CERTS add-not-replace semantics).
// We have custom anchors: include system anchors so that private CAs
// AUGMENT rather than REPLACE system trust (mirrors Node's
// NODE_EXTRA_CA_CERTS add-not-replace semantics). The optional
// `systemCaPem` parameter is a test injection point: `undefined`
// means "read the platform bundle" (production), a string means
// "use this controlled fixture" (test), and `null` means "explicitly
// skip" (negative-control test).
let systemCa: string | null = null;
if (systemCaPem === null) {
// Explicit skip: negative-control path.
} else if (systemCaPem !== undefined) {
systemCa = validateCaBundlePem(systemCaPem);
} else {
systemCa = await readSystemCaBundle();
}
if (systemCa) customChunks.unshift(systemCa);
const target = path.join(metaDir, COMBINED_FILENAME);
const body = `${customChunks.join('\n')}\n`;
await fs.writeFile(target, body, { mode: 0o600 });
return { path: target.split(path.sep).join('/') };
}
+201 -104
View File
@@ -5,10 +5,15 @@ import path from 'path';
import { ensureGitBinary, getGitExecPath } from './gitBinary';
import {
CREDENTIAL_HELPER_CONFIG_VALUE,
GIT_ALLOWED_HOST_ENV_VAR,
GIT_HELPER_PATH_ENV_VAR,
GIT_TOKEN_ENV_VAR,
writeCredentialHelper,
} from './credentialHelper';
import { credentialScopeHost } from './caBundle';
import { sanitizeForLog } from '../../utils/safeLog';
import { writeCombinedCaBundle } from './gitCaBundleSink';
import { looksLikeRedirectFailure, resolveRedirectedRepoUrl } from './redirectPreflight';
import { isTransportFailure, type TransportFailure } from './errors';
import type { FetchRequest, FetchResult, GitTransport, ResolveRequest, ResolveResult } from './types';
import {
@@ -143,7 +148,7 @@ async function awaitKillConfirmed(kill: Promise<void> | undefined, what: string)
let timer: NodeJS.Timeout | undefined;
const bound = new Promise<void>((resolve) => {
timer = setTimeout(() => {
console.warn(`[GitSource:transport] ${what} not confirmed within ${KILL_CONFIRM_TIMEOUT_MS}ms; continuing cleanup while it may still be running`);
console.warn(`[GitSource:transport] ${sanitizeForLog(what)} not confirmed within ${KILL_CONFIRM_TIMEOUT_MS}ms; continuing cleanup while it may still be running`);
resolve();
}, KILL_CONFIRM_TIMEOUT_MS);
});
@@ -272,6 +277,7 @@ function buildEnv(
token?: string | null,
helperPath?: string | null,
sshCommand?: string | null,
allowedHost?: string | null,
): NodeJS.ProcessEnv {
const env: NodeJS.ProcessEnv = {
...process.env,
@@ -301,6 +307,9 @@ function buildEnv(
// parses it. See credentialHelper.ts.
env[GIT_HELPER_PATH_ENV_VAR] = helperPath;
}
if (allowedHost) {
env[GIT_ALLOWED_HOST_ENV_VAR] = allowedHost;
}
if (sshCommand) {
env.GIT_SSH_COMMAND = sshCommand;
}
@@ -331,85 +340,61 @@ async function detectWindowsCABundle(): Promise<string | null> {
return null;
}
/** First existing system CA bundle for OpenSSL-backed git on POSIX. */
const POSIX_CA_BUNDLE_CANDIDATES = [
'/etc/ssl/certs/ca-certificates.crt',
'/etc/pki/tls/certs/ca-bundle.crt',
];
/**
* Build the CA-anchor configuration for one fetch.
*
* Mirrors Node's own NODE_EXTRA_CA_CERTS semantics (extra anchors ADDED to
* the defaults, never replacing them) by writing a combined PEM bundle into
* the fetch workspace's `.meta` dir:
* - No NODE_EXTRA_CA_CERTS: production posture. POSIX passes nothing and
* lets OpenSSL use system trust; Windows pins Git's own bundled bundle,
* because stripping system gitconfig also strips the installer's pointer
* to it.
* - With NODE_EXTRA_CA_CERTS: defaults PLUS the extra CAs, so the dev/E2E
* fixture server and public hosts validate in the same process state.
* - No NODE_EXTRA_CA_CERTS and no per-source PEM: production posture.
* POSIX passes nothing and lets OpenSSL use system trust; Windows pins
* Git's own bundled bundle, because stripping system gitconfig also
* strips the installer's pointer to it.
* - With NODE_EXTRA_CA_CERTS and/or a per-source PEM: defaults PLUS the extra
* CAs, so private-CA servers and public hosts validate in the same fetch.
*
* The per-source PEM and the env-var file are written by
* `writeCombinedCaBundle` in `./gitCaBundleSink.ts`; the sink module is the
* single point CodeQL is asked to ignore for `js/http-to-file-access` because
* every input it writes is either a file path inside the per-fetch workspace
* (no external taint) or a PEM that the caller has already validated.
*/
async function resolveCaArgs(layout: WorkspaceLayout): Promise<string[]> {
const extraPath = process.env.NODE_EXTRA_CA_CERTS;
const hasExtra = Boolean(extraPath && existsSync(extraPath));
async function resolveCaArgs(
layout: WorkspaceLayout,
perSourceCaPem?: string | null,
): Promise<{ args: string[]; caPath: string | null }> {
const isWindows = process.platform === 'win32';
if (!hasExtra && !isWindows) {
return [];
// Per-source and env-var anchors live in a single combined file written by
// the sink module. The sink now includes system anchors when custom
// anchors are present, so we only need to handle the "no custom anchors"
// case specially for Windows.
const combined = await writeCombinedCaBundle(layout.metaDir, perSourceCaPem);
if (combined) {
return { args: ['-c', `http.sslCAInfo=${combined.path}`], caPath: combined.path };
}
if (isWindows && !hasExtra) {
// Windows without an override: anchor to Git's bundled bundle directly.
const bundle = await detectWindowsCABundle();
return bundle ? ['-c', `http.sslCAInfo=${bundle}`] : [];
}
let defaultPem = '';
let winBundle: string | null = null;
// No custom anchors: on POSIX we pass nothing (system trust applies
// directly via OpenSSL). On Windows we still need the Git-bundled
// pointer because GIT_CONFIG_NOSYSTEM stripped the installer's config.
if (isWindows) {
winBundle = await detectWindowsCABundle();
if (winBundle) {
try {
defaultPem = await fs.readFile(winBundle.replace(/\//g, path.sep), 'utf8');
} catch {
console.warn(`[GitSource:transport] could not read system CA bundle at ${winBundle}; combined anchors will contain only NODE_EXTRA_CA_CERTS entries.`);
}
}
} else {
for (const candidate of POSIX_CA_BUNDLE_CANDIDATES) {
if (!existsSync(candidate)) continue;
try {
defaultPem = await fs.readFile(candidate, 'utf8');
break;
} catch {
// Try the next candidate.
}
}
if (!defaultPem) {
console.warn('[GitSource:transport] no readable system CA bundle found; combined anchors will contain only NODE_EXTRA_CA_CERTS entries.');
}
const bundle = await detectWindowsCABundle();
return bundle ? { args: ['-c', `http.sslCAInfo=${bundle}`], caPath: bundle } : { args: [], caPath: null };
}
let extraPem = '';
try {
extraPem = await fs.readFile(extraPath as string, 'utf8');
} catch {
console.warn('[GitSource:transport] could not read the file configured via NODE_EXTRA_CA_CERTS; ignoring custom anchors.');
// Windows still has working defaults; fall back to them instead of
// dropping every anchor.
return isWindows && winBundle ? ['-c', `http.sslCAInfo=${winBundle}`] : [];
}
const combinedPath = path.join(layout.metaDir, 'combined-ca.pem');
await fs.writeFile(combinedPath, `${defaultPem}\n${extraPem}`, { mode: 0o600 });
return ['-c', `http.sslCAInfo=${combinedPath.split(path.sep).join('/')}`];
return { args: [], caPath: null };
}
/**
* Config shared by every invocation. With no helper, credential.helper is
* explicitly cleared so nothing from the environment can answer prompts.
*/
async function commonArgs(layout: WorkspaceLayout, helperPath: string | null, ssh: boolean): Promise<string[]> {
async function commonArgs(
layout: WorkspaceLayout,
helperPath: string | null,
ssh: boolean,
perSourceCaPem?: string | null,
): Promise<{ args: string[]; caPath: string | null }> {
const args = [
'-c', 'protocol.allow=never',
];
@@ -417,6 +402,14 @@ async function commonArgs(layout: WorkspaceLayout, helperPath: string | null, ss
args.push('-c', 'protocol.ssh.allow=always');
} else {
args.push('-c', 'protocol.https.allow=always');
// Git never follows a redirect itself, so it can never contact a
// destination this process has not already approved. When a server
// does redirect, the caller walks the chain unauthenticated through
// redirectPreflight, validates every hop, and re-runs git against the
// approved URL. Letting git follow instead would contact the target
// before any policy ran, which is what makes the internal-range guard
// meaningful rather than after-the-fact.
args.push('-c', 'http.followRedirects=false');
}
args.push('-c', `core.hooksPath=${layout.hooksDir.split(path.sep).join('/')}`);
if (process.platform === 'win32') {
@@ -428,7 +421,8 @@ async function commonArgs(layout: WorkspaceLayout, helperPath: string | null, ss
// git is OpenSSL-backed and unaffected by this flag's absence.
args.push('-c', 'http.sslBackend=openssl');
}
args.push(...await resolveCaArgs(layout));
const ca = await resolveCaArgs(layout, perSourceCaPem);
args.push(...ca.args);
if (helperPath !== null) {
// A fixed value: the helper's path reaches git through the child env
// instead of being interpolated here, so a workspace path containing
@@ -438,7 +432,7 @@ async function commonArgs(layout: WorkspaceLayout, helperPath: string | null, ss
} else {
args.push('-c', 'credential.helper=');
}
return args;
return { args, caPath: ca.caPath };
}
/**
@@ -455,9 +449,11 @@ async function commonArgs(layout: WorkspaceLayout, helperPath: string | null, ss
*/
async function prepareInvocation(
workspaceRoot: string,
repoUrl: string,
token?: string | null,
sshAuth?: ResolveRequest['sshAuth'],
): Promise<{ layout: WorkspaceLayout; env: NodeJS.ProcessEnv; baseArgs: string[] }> {
caBundlePem?: string | null,
): Promise<{ layout: WorkspaceLayout; env: NodeJS.ProcessEnv; baseArgs: string[]; allowedHost: string | null; caPath: string | null }> {
const layout = await prepareWorkspace(workspaceRoot);
let sshCommand: string | null = null;
if (sshAuth) {
@@ -466,9 +462,63 @@ async function prepareInvocation(
sshCommand = buildSshCommand(keyPath, knownPath);
}
const helperPath = token ? await writeCredentialHelper(layout.metaDir) : null;
const env = buildEnv(layout.homeDir, token, helperPath, sshCommand);
const baseArgs = await commonArgs(layout, helperPath, Boolean(sshAuth));
return { layout, env, baseArgs };
const parsed = parseRepoTransportUrl(repoUrl);
const allowedHost = parsed?.kind === 'https' && token
? credentialScopeHost(parsed.host)
: null;
const env = buildEnv(layout.homeDir, token, helperPath, sshCommand, allowedHost);
const { args: baseArgs, caPath } = await commonArgs(layout, helperPath, Boolean(sshAuth), caBundlePem);
return { layout, env, baseArgs, allowedHost, caPath };
}
/**
* The redirect chain for a repository, resolved and policy-checked without
* credentials, or null when the source does not redirect (in which case the
* caller keeps git's original failure). Only ever consulted after git has
* already refused to follow a redirect, so the normal path costs nothing.
*/
async function approveRedirectTarget(
repo: ParsedRepoUrl,
stderr: string,
hasToken: boolean,
caPath: string | null,
): Promise<string | null> {
if (repo.kind !== 'https' || !looksLikeRedirectFailure(stderr)) return null;
let caPem: string | undefined;
if (caPath) {
try {
caPem = await fs.readFile(caPath, 'utf8');
} catch (e) {
// This bundle was written moments ago by this same invocation, so a
// read failure is a real fault, not a missing option. Probing with
// default trust instead would validate the operator's private-CA
// host against the wrong anchors, so decline the retry and say so.
console.warn(`[GitSource:transport] could not read the CA bundle at ${sanitizeForLog(caPath)} for the redirect preflight; not authorising a redirect retry: ${sanitizeForLog(e instanceof Error ? e.message : String(e))}`);
return null;
}
}
return await resolveRedirectedRepoUrl({
repoUrl: repo.href,
hasToken,
reportHost: repoHostLabel(repo),
caPem,
});
}
/**
* The same question asked of a materialization step, which reports through a
* thrown TransportFailure rather than an exit code. Kept in one place so the
* rule for which failures may be retried cannot drift between the fetch and
* fast-forward paths.
*/
async function approvedRedirectForError(
e: unknown,
repo: ParsedRepoUrl,
hasToken: boolean,
caPath: string | null,
): Promise<string | null> {
if (!isTransportFailure(e) || e.reason !== 'exit' || !e.stderr) return null;
return await approveRedirectTarget(repo, e.stderr, hasToken, caPath);
}
// ─── Input validation ────────────────────────────────────────────────────────
@@ -635,19 +685,30 @@ async function lsRemoteRefs(
baseArgs: string[],
timeoutMs: number,
hasToken: boolean,
caPath: string | null = null,
): Promise<ResolvedRemoteRefs> {
const host = repoHostLabel(repo);
let res: RunResult;
try {
res = await runGit(
[...baseArgs, 'ls-remote', repo.href, `refs/heads/${ref}`, `refs/tags/${ref}`, `refs/tags/${ref}^{}`],
{ env, timeoutMs: Math.min(timeoutMs, LS_REMOTE_MAX_MS) },
);
} catch (e) {
if (isTimeoutError(e)) {
throw { transportFailure: true as const, reason: 'timeout', host, hasToken } satisfies TransportFailure;
const attempt = async (href: string): Promise<RunResult> => {
try {
return await runGit(
[...baseArgs, 'ls-remote', href, `refs/heads/${ref}`, `refs/tags/${ref}`, `refs/tags/${ref}^{}`],
{ env, timeoutMs: Math.min(timeoutMs, LS_REMOTE_MAX_MS) },
);
} catch (e) {
if (isTimeoutError(e)) {
throw { transportFailure: true as const, reason: 'timeout', host, hasToken } satisfies TransportFailure;
}
throw e;
}
throw e;
};
let res = await attempt(repo.href);
if (res.exitCode !== 0) {
// git refused a redirect. Resolve and approve the destination first;
// an approved chain is retried once against the final URL, and a
// rejected one throws before that host is ever contacted.
const approved = await approveRedirectTarget(repo, res.stderr, hasToken, caPath);
if (approved) res = await attempt(approved);
}
if (res.exitCode !== 0) {
throw { transportFailure: true as const, reason: 'exit', stderr: res.stderr, exitCode: res.exitCode, argv: baseArgs, host, hasToken } satisfies TransportFailure;
@@ -692,6 +753,7 @@ export async function verifyFastForward(req: {
descendantSha: string;
token?: string | null;
sshAuth?: ResolveRequest['sshAuth'];
caBundlePem?: string | null;
timeoutMs?: number;
workspaceRoot: string;
maxBytes: number;
@@ -712,7 +774,12 @@ export async function verifyFastForward(req: {
throw { transportFailure: true as const, reason: 'timeout', host, hasToken } satisfies TransportFailure;
}
};
const { env, baseArgs } = await prepareInvocation(req.workspaceRoot, req.token, req.sshAuth);
const { env, baseArgs, caPath } = await prepareInvocation(
req.workspaceRoot, req.repoUrl, req.token, req.sshAuth, req.caBundlePem,
);
// Resolved once if the host refuses a redirect, then reused by the deepen
// rounds so they do not each re-walk the same chain.
let effectiveHref = repo.href;
const repoDir = path.join(req.workspaceRoot, 'ff-check');
await fs.mkdir(repoDir, { recursive: true });
@@ -803,7 +870,14 @@ export async function verifyFastForward(req: {
};
await materialize([...baseArgs, 'init']);
await materialize([...baseArgs, 'fetch', '--depth=1', repo.href, descendant]);
try {
await materialize([...baseArgs, 'fetch', '--depth=1', effectiveHref, descendant]);
} catch (e) {
const approved = await approvedRedirectForError(e, repo, hasToken, caPath);
if (!approved) throw e;
effectiveHref = approved;
await materialize([...baseArgs, 'fetch', '--depth=1', effectiveHref, descendant]);
}
const countReachable = async (): Promise<number> => {
const argv = [...baseArgs, 'rev-list', '--count', descendant];
@@ -891,7 +965,7 @@ export async function verifyFastForward(req: {
}
const previousCount = reachableCount;
await materialize([...baseArgs, 'fetch', `--deepen=${deepenStep}`, repo.href, descendant]);
await materialize([...baseArgs, 'fetch', `--deepen=${deepenStep}`, effectiveHref, descendant]);
fetchRounds += 1;
reachableCount = await countReachable();
@@ -929,10 +1003,13 @@ export const nativeGitTransport: GitTransport = {
}
assertValidRef(req.ref, repoHostLabel(repo), hasToken);
const { env, baseArgs } = await prepareInvocation(req.workspaceRoot, req.token, req.sshAuth);
const { env, baseArgs, caPath } = await prepareInvocation(
req.workspaceRoot, req.repoUrl, req.token, req.sshAuth, req.caBundlePem,
);
const found = await lsRemoteRefs(
repo, req.ref, env, baseArgs,
req.timeoutMs ?? DEFAULT_TIMEOUT_MS, hasToken,
req.timeoutMs ?? DEFAULT_TIMEOUT_MS, hasToken, caPath,
);
if (found.branchSha) return { commitSha: found.branchSha, kind: 'branch' };
if (found.tagSha) return { commitSha: found.tagSha, kind: 'tag' };
@@ -945,7 +1022,13 @@ export const nativeGitTransport: GitTransport = {
const repo = assertValidRepoUrl(req.repoUrl, hasToken);
assertValidRef(req.ref, repoHostLabel(repo), hasToken);
const { layout, env, baseArgs } = await prepareInvocation(req.workspaceRoot, req.token, req.sshAuth);
// The credential scope host is set inside prepareInvocation via
// GIT_ALLOWED_HOST_ENV_VAR so the credential helper refuses to emit
// credentials for any other host. A refused redirect is resolved and
// approved by the preflight below before any retry.
const { layout, env, baseArgs, caPath } = await prepareInvocation(
req.workspaceRoot, req.repoUrl, req.token, req.sshAuth, req.caBundlePem,
);
const checkout = path.join(req.workspaceRoot, 'repo');
const timeoutMs = req.timeoutMs ?? DEFAULT_TIMEOUT_MS;
@@ -998,28 +1081,42 @@ export const nativeGitTransport: GitTransport = {
return res;
};
if (req.refKind === 'sha') {
// `--branch` cannot take a bare SHA, so a pinned commit uses a
// third strategy: init a repo, fetch exactly that object, and
// check it out detached. The host must allow fetching a direct
// SHA (GitHub does by default); a refusal surfaces as a
// non-zero `git fetch` here and classifies as UNSUPPORTED_REF.
await materialize([...baseArgs, 'init', checkout]);
await materialize([...baseArgs, '-C', checkout, 'fetch', '--depth=1', repo.href, req.ref]);
await materialize([...baseArgs, '-C', checkout, 'checkout', '--detach', req.ref]);
} else {
// A bare name works for both branches and tags: `--branch`
// detaches at the named ref's commit either way, and passing a
// fully-qualified `refs/tags/<ref>` is rejected by git
// (`Remote branch ... not found`). The resolved kind is
// already pinned by ls-remote, and the rev-parse HEAD
// verification below confirms the checkout matched it.
const branchArg = req.ref;
await materialize([
...baseArgs, 'clone',
'--depth=1', '--single-branch', '--no-tags', '--no-recurse-submodules',
'--branch', branchArg, repo.href, checkout,
]);
const runMaterialization = async (href: string): Promise<void> => {
if (req.refKind === 'sha') {
// `--branch` cannot take a bare SHA, so a pinned commit uses a
// third strategy: init a repo, fetch exactly that object, and
// check it out detached. The host must allow fetching a direct
// SHA (GitHub does by default); a refusal surfaces as a
// non-zero `git fetch` here and classifies as UNSUPPORTED_REF.
await materialize([...baseArgs, 'init', checkout]);
await materialize([...baseArgs, '-C', checkout, 'fetch', '--depth=1', href, req.ref]);
await materialize([...baseArgs, '-C', checkout, 'checkout', '--detach', req.ref]);
} else {
// A bare name works for both branches and tags: `--branch`
// detaches at the named ref's commit either way, and passing a
// fully-qualified `refs/tags/<ref>` is rejected by git
// (`Remote branch ... not found`). The resolved kind is
// already pinned by ls-remote, and the rev-parse HEAD
// verification below confirms the checkout matched it.
await materialize([
...baseArgs, 'clone',
'--depth=1', '--single-branch', '--no-tags', '--no-recurse-submodules',
'--branch', req.ref, href, checkout,
]);
}
};
try {
await runMaterialization(repo.href);
} catch (e) {
// A refused redirect is the one failure worth a second attempt,
// and only against a destination the preflight has approved.
const approved = await approvedRedirectForError(e, repo, hasToken, caPath);
if (!approved) throw e;
// The refused attempt can have left a partial checkout behind;
// git refuses to clone into a non-empty directory.
await fs.rm(checkout, { recursive: true, force: true });
await runMaterialization(approved);
}
let actual: string;
@@ -0,0 +1,203 @@
import https from 'https';
import type { TransportFailure } from './errors';
import { credentialScopeHost } from './caBundle';
import { sanitizeForLog } from '../../utils/safeLog';
/**
* Destination-aware redirect policy for HTTPS Git operations.
*
* Git is always run with `http.followRedirects=false`, so it never contacts a
* redirect target on its own. When git refuses a redirect, this module walks
* the chain itself with an UNAUTHENTICATED request and validates every hop
* before anything follows it. Only a chain that satisfies the policy end to
* end produces a URL git is then re-run against.
*
* The ordering is the point: a destination outside the configured
* repository's origin is rejected while it has still never been contacted, so
* a hostile server cannot use a redirect either to move a credential or to
* turn a repository fetch into a probe of a host it chose. Parsing git's own
* output cannot achieve this, because git prints the destination only on the
* path where it has already followed the redirect (`warning: redirecting to
* <url>` in git-remote-http) and prints nothing at all when following is
* disabled.
*
* The rule every hop must satisfy is deliberately one rule: the destination
* stays on the configured repository's origin (scheme, host, and port), and
* only the path may move. That is what a repository relocating to a canonical
* path looks like, it is what git's own `update_url_from_redirect` superset
* check enforces on the path component, and it makes a redirect to an
* internal address structurally impossible rather than something a separate
* address-range blocklist has to anticipate. Such a blocklist would also be
* actively wrong here: a self-hosted Git server on loopback or a private LAN
* range is a supported deployment, not an attack.
*/
/** Hop ceiling for one chain, bounding both loops and probe cost. */
export const MAX_REDIRECT_HOPS = 5;
/** The smart-HTTP endpoint whose redirect defines where the repository moved. */
const REF_ADVERTISE_SUFFIX = '/info/refs';
const REF_ADVERTISE_QUERY = 'service=git-upload-pack';
const PROBE_TIMEOUT_MS = 10_000;
function redirectScope(host: string, hasToken: boolean): TransportFailure {
return { transportFailure: true as const, reason: 'redirect-scope', host, hasToken };
}
/** Parse a Location value (absolute or relative) against its base. Null on failure, so callers fail closed. */
export function resolveLocation(baseUrl: string, location: string): string | null {
try {
return new URL(location, baseUrl).toString();
} catch {
return null;
}
}
/**
* The origin a redirect is allowed to stay on, in the same `host[:port]`
* spelling the credential helper compares against, so the transport's
* redirect rule and its credential rule cannot drift apart.
*/
export function redirectScopeOf(url: string): string | null {
try {
const parsed = new URL(url);
if (parsed.protocol !== 'https:') return null;
return credentialScopeHost(parsed.hostname, parsed.port ? Number(parsed.port) : undefined);
} catch {
return null;
}
}
/**
* True when git's stderr describes a refused redirect rather than an ordinary
* failure. Matched against git's current wording, which
* `git-redirect-preflight.test.ts` pins to the literal strings git emits, so a
* git upgrade that rephrases them fails a test rather than quietly making
* relocated repositories unreachable. A miss is safe but not silent: no retry
* is authorised and git's own error is reported unchanged.
*/
export function looksLikeRedirectFailure(stderr: string): boolean {
return /returned error: 30\d/i.test(stderr) || /\bredirect/i.test(stderr);
}
/**
* The single gate every URL passes before it is requested. Returns the URL
* only when it sits on `expectedScope`, and throws otherwise, so no caller can
* reach the network with a destination that has not been checked. The seed URL
* goes through it too: that check is trivially true, but routing every request
* through one place is what makes the guarantee inspectable rather than a
* property of the loop's shape, for a reader as much as for static analysis.
*/
export function approvedUrl(
url: string,
expectedScope: string,
reportHost: string,
hasToken: boolean,
): string {
if (redirectScopeOf(url) !== expectedScope) {
throw redirectScope(reportHost, hasToken);
}
return url;
}
interface ProbeResponse {
status: number;
location: string | null;
}
/** One unauthenticated GET, following nothing. Rejects only on transport errors. */
function probe(url: string, ca: string | undefined): Promise<ProbeResponse> {
return new Promise((resolve, reject) => {
const req = https.get(url, { ca, timeout: PROBE_TIMEOUT_MS }, (res) => {
const location = typeof res.headers.location === 'string' ? res.headers.location : null;
// The body is irrelevant; discard it so the socket can close.
res.resume();
resolve({ status: res.statusCode ?? 0, location });
});
req.on('timeout', () => req.destroy(new Error('redirect preflight timed out')));
req.on('error', reject);
});
}
/**
* Strip the ref-advertise suffix off a probe URL to recover the repository
* URL git should be pointed at. A destination that no longer ends in the
* endpoint we asked for is not a relocation of this repository and is
* refused, mirroring git's own superset rule on the path component.
*/
function repoUrlFromProbeUrl(probeUrl: string): string | null {
let parsed: URL;
try {
parsed = new URL(probeUrl);
} catch {
return null;
}
if (!parsed.pathname.endsWith(REF_ADVERTISE_SUFFIX)) return null;
parsed.pathname = parsed.pathname.slice(0, -REF_ADVERTISE_SUFFIX.length);
parsed.search = '';
parsed.hash = '';
return parsed.toString().replace(/\/$/, '');
}
/**
* Build the ref-advertise probe URL for `repoUrl` via the `URL` API rather
* than string concatenation, so a `repoUrl` that already carries a query
* string (a signed URL, say) gets the suffix inserted into the path and the
* query replaced, instead of the suffix landing after the existing query.
*/
function refAdvertiseProbeUrl(repoUrl: string): string {
const parsed = new URL(repoUrl);
parsed.pathname = `${parsed.pathname.replace(/\/$/, '')}${REF_ADVERTISE_SUFFIX}`;
parsed.search = `?${REF_ADVERTISE_QUERY}`;
return parsed.toString();
}
/**
* Walk the redirect chain for `repoUrl` without credentials and return the
* repository URL it ultimately resolves to, or null when the source does not
* redirect at all (so the caller keeps git's original failure).
*
* Throws a `redirect-scope` TransportFailure as soon as a hop leaves the
* configured origin, before that hop is ever requested.
*/
export async function resolveRedirectedRepoUrl(opts: {
repoUrl: string;
hasToken: boolean;
reportHost: string;
caPem?: string;
}): Promise<string | null> {
const expectedScope = redirectScopeOf(opts.repoUrl);
if (!expectedScope) throw redirectScope(opts.reportHost, opts.hasToken);
let current = approvedUrl(
refAdvertiseProbeUrl(opts.repoUrl),
expectedScope, opts.reportHost, opts.hasToken,
);
let hops = 0;
while (hops < MAX_REDIRECT_HOPS) {
let res: ProbeResponse;
try {
res = await probe(current, opts.caPem);
} catch (e) {
// The probe could not complete (TLS, DNS, reset). We cannot prove
// the chain is safe, so we do not authorise a retry; the caller
// reports git's original error instead. Say why, or a private CA
// that fails to validate is indistinguishable from a server that
// simply does not redirect.
console.warn(`[GitSource:redirect] could not probe ${sanitizeForLog(opts.reportHost)} for a redirect target, keeping the original git error: ${sanitizeForLog(e instanceof Error ? e.message : String(e))}`);
return null;
}
if (res.status < 300 || res.status >= 400 || !res.location) {
if (hops === 0) return null;
const resolved = repoUrlFromProbeUrl(current);
if (!resolved) throw redirectScope(opts.reportHost, opts.hasToken);
return resolved;
}
const next = resolveLocation(current, res.location);
if (!next) throw redirectScope(opts.reportHost, opts.hasToken);
current = approvedUrl(next, expectedScope, opts.reportHost, opts.hasToken);
hops += 1;
}
throw redirectScope(opts.reportHost, opts.hasToken);
}
+2
View File
@@ -30,6 +30,8 @@ export interface ResolveRequest {
ref: string;
token?: string | null;
sshAuth?: SshDeployKeyAuth | null;
/** Optional per-source custom CA PEM bundle (system anchors are still included). */
caBundlePem?: string | null;
/**
* Total fetch budget in milliseconds. Note: the resolution round trip
* (ls-remote) is internally capped at 10s regardless of this value, so
@@ -267,6 +267,7 @@ async function resolveOne(checkpoint: GitOpsCreateCheckpointRow): Promise<Create
encrypted_deploy_key: checkpoint.encrypted_deploy_key,
ssh_known_hosts_entry: checkpoint.ssh_known_hosts_entry,
ssh_host_key_fingerprint: checkpoint.ssh_host_key_fingerprint,
encrypted_ca_bundle: checkpoint.encrypted_ca_bundle,
auto_apply_on_webhook: checkpoint.auto_apply_on_webhook === 1,
auto_deploy_on_apply: checkpoint.auto_deploy_on_apply === 1,
last_applied_commit_sha: checkpoint.commit_sha,
@@ -231,6 +231,7 @@ export function buildCreateCheckpointRow(args: {
encryptedDeployKey?: string | null;
sshKnownHostsEntry?: string | null;
sshHostKeyFingerprint?: string | null;
encryptedCaBundle?: string | null;
autoApplyOnWebhook: boolean;
autoDeployOnApply: boolean;
commitSha: string;
@@ -257,6 +258,7 @@ export function buildCreateCheckpointRow(args: {
encrypted_deploy_key: args.encryptedDeployKey ?? null,
ssh_known_hosts_entry: args.sshKnownHostsEntry ?? null,
ssh_host_key_fingerprint: args.sshHostKeyFingerprint ?? null,
encrypted_ca_bundle: args.encryptedCaBundle ?? null,
auto_apply_on_webhook: args.autoApplyOnWebhook ? 1 : 0,
auto_deploy_on_apply: args.autoDeployOnApply ? 1 : 0,
commit_sha: args.commitSha,
+1
View File
@@ -34,6 +34,7 @@ CREATE TABLE IF NOT EXISTS gitops_create_checkpoints (
encrypted_deploy_key TEXT NULL,
ssh_known_hosts_entry TEXT NULL,
ssh_host_key_fingerprint TEXT NULL,
encrypted_ca_bundle TEXT NULL,
auto_apply_on_webhook INTEGER NOT NULL DEFAULT 0,
auto_deploy_on_apply INTEGER NOT NULL DEFAULT 0,
commit_sha TEXT NULL,
+4 -2
View File
@@ -266,14 +266,16 @@ export class GitOpsStore {
application_id, stack_name, phase, generation_id, operation_id, repo_url, branch,
compose_path, compose_paths_json, context_dir, sync_env, env_path, auth_type,
encrypted_token, encrypted_deploy_key, ssh_known_hosts_entry, ssh_host_key_fingerprint,
encrypted_ca_bundle,
auto_apply_on_webhook, auto_deploy_on_apply, commit_sha,
applied_spec_json, created_managed_root, created_at, updated_at
) VALUES (${Array(24).fill('?').join(', ')})`,
) VALUES (${Array(25).fill('?').join(', ')})`,
).run(
row.application_id, row.stack_name, row.phase, row.generation_id, row.operation_id,
row.repo_url, row.branch, row.compose_path, row.compose_paths_json, row.context_dir,
row.sync_env, row.env_path, row.auth_type, row.encrypted_token, row.encrypted_deploy_key,
row.ssh_known_hosts_entry, row.ssh_host_key_fingerprint, row.auto_apply_on_webhook,
row.ssh_known_hosts_entry, row.ssh_host_key_fingerprint, row.encrypted_ca_bundle,
row.auto_apply_on_webhook,
row.auto_deploy_on_apply, row.commit_sha, row.applied_spec_json, row.created_managed_root,
row.created_at, row.updated_at,
);
+1
View File
@@ -123,6 +123,7 @@ export type GitOpsCreateCheckpointRow = {
encrypted_deploy_key: string | null;
ssh_known_hosts_entry: string | null;
ssh_host_key_fingerprint: string | null;
encrypted_ca_bundle: string | null;
auto_apply_on_webhook: number;
auto_deploy_on_apply: number;
commit_sha: string | null;