docs(git): publish a versioned Git transport support matrix (#1883)

* fix(git): make gitSourceStatus exhaustive over GitSourceErrorCode

GIT_ERROR was the only code falling through the implicit default
branch. Give it an explicit case and add the same never-guard
webhookPullStatus already uses, so a future code with no mapping is a
compile error instead of a silent 400.

* fix(git): fail loudly under CI when git or sshd is missing

Every real-git and real-sshd integration suite carried its own local
gitAvailable()/sshdAvailable() probe and skipped silently when the
dependency was absent, in CI as well as locally. A cell in the
upcoming support matrix could then advertise automated proof while
the test that proves it never ran.

Consolidate into shared requireGitBinary()/requireSshd() helpers
(one for backend vitest, one for Playwright, since backend's rootDir
pin blocks a cross-directory import) that take an injectable probe.
Locally a missing dependency still skips; under CI it throws with an
actionable message naming what's missing.

* feat(docs): publish a versioned Git transport support matrix

Adds docs/git-transport-support.yaml as the canonical claim set for
every transport/ref/auth/host/CA combination Git Sources supports,
each claim naming its own reproducible evidence rather than
generalizing from a related test. A claim is supported only when a
real end-to-end test (or a dated live attestation) proves that exact
combination; everything else is marked unverified, never assumed.

The published page (docs/features/git-transport-support.mdx) is
generated from the YAML by backend/scripts/git-support-matrix, so it
cannot silently drift from what the tests actually prove. A new
backend test (git-support-matrix.test.ts) enforces this: schema
validity, evidence semantics (supported needs success evidence,
unsupported needs a reproducible rejection, unverified forbids
evidence entirely), byte-identical page generation, and that every
referenced test title resolves via the TypeScript AST rather than a
string search that a skipped or commented-out test would pass.

The error-model section is cross-checked against the real
GitSourceErrorCode and TransportFacingCode unions and against
gitSourceStatus's actual HTTP mapping, so the matrix and the runtime
behavior cannot diverge either.

Named Git hosts (GitHub, GitLab, Gitea, Forgejo, Bitbucket) and the
direct-proxy/Pilot execution paths are seeded as unverified pending a
live attestation pass; only the generic local-fixture combinations
already proven by the real-git integration suites are marked
supported today.

* docs(git): scope GitHub claims to what this pass can actually attest

Splits the GitHub row into a public no-auth claim (attestable with a
real public repository) and separate PAT/SSH deploy-key claims marked
unverified with an explicit reason: this pass holds no real GitHub
credential to exercise them with, and none is assumed or fabricated.

* feat(docs): attest the Git transport matrix live against real hosts

Runs the QA fleet's live Sencho instance through the transport
combinations that automated fixtures cannot exercise, then records
each result in docs/git-transport-attestations.yaml so it can be
re-run and compared later.

GitHub, GitLab, and Bitbucket are attested over public HTTPS against
real, stable, publicly-owned demo repositories (branch and pinned
SHA; GitLab additionally has a tagged fixture). Gitea and Forgejo get
full coverage (branch, tag, and SHA, over both HTTPS with a
per-source CA and SSH with a deploy key) against disposable
self-hosted instances stood up for this pass, including a private
repository so the authentication and host-key failure classifiers
were exercised against a real wrong credential and a real wrong host
key, not just the mocked corpus. The direct-proxy and Pilot execution
paths are each confirmed once against a real public host, proving
the distributed dispatch itself rather than assuming it from the
local-path evidence.

Left honestly unverified: GitHub PAT and SSH deploy-key auth (this
pass holds no real GitHub credential), a GitHub tag combination (no
small stable tagged fixture found), and a Bitbucket tag combination
(the fixture repository carries none). Every claim's evidence records
its exact transport, ref, auth, host, CA, and node path so nothing
here is extrapolated from a neighboring result.

All infrastructure created for this pass (two throwaway Git server
containers, one probe stack) was torn down afterward and the fleet's
container list was confirmed to match its state before the pass.

* style(git): replace em dashes and fix a stale .mjs reference

Directive 18 applies to code comments and build markers too, not just
prose. Also corrects the claim set's header comment, which still
named render.mjs after the renderer was moved to render.js to match
the house convention for backend scripts.
This commit is contained in:
Anso
2026-09-01 21:56:02 -04:00
committed by GitHub
parent 851a5fb41e
commit 0928765232
24 changed files with 2221 additions and 34 deletions
+49
View File
@@ -0,0 +1,49 @@
/**
* Unit coverage for the shared dependency probes in externalDeps.ts.
*
* Not a browser test: these run against the probe functions directly with an
* injected predicate, so absence of git/sshd can be exercised without
* removing system binaries.
*/
import { test, expect } from '@playwright/test';
import { requireGitBinary, requireSshd } from './externalDeps';
test.describe('external dependency probes', () => {
test.afterEach(() => {
delete process.env.CI;
});
test('requireGitBinary returns true when git is present, locally or in CI', () => {
delete process.env.CI;
expect(requireGitBinary(() => true)).toBe(true);
process.env.CI = '1';
expect(requireGitBinary(() => true)).toBe(true);
});
test('requireGitBinary returns false when git is absent locally', () => {
delete process.env.CI;
expect(requireGitBinary(() => false)).toBe(false);
});
test('requireGitBinary throws when git is absent under CI', () => {
process.env.CI = '1';
expect(() => requireGitBinary(() => false)).toThrow(/git is required in CI/);
});
test('requireSshd returns true when sshd is present, locally or in CI', () => {
delete process.env.CI;
expect(requireSshd(() => true)).toBe(true);
process.env.CI = '1';
expect(requireSshd(() => true)).toBe(true);
});
test('requireSshd returns false when sshd is absent locally', () => {
delete process.env.CI;
expect(requireSshd(() => false)).toBe(false);
});
test('requireSshd throws when sshd is absent under CI', () => {
process.env.CI = '1';
expect(() => requireSshd(() => false)).toThrow(/sshd is required in CI/);
});
});
+49
View File
@@ -0,0 +1,49 @@
/**
* Shared availability probes for the real-git and real-sshd E2E fixtures.
*
* Mirrors backend/src/__tests__/__helpers__/externalDeps.ts. Kept as a
* separate file rather than a shared import: backend's tsconfig pins
* `rootDir` to backend/src, so a cross-directory import would fail
* `tsc --noEmit` there.
*
* `gitServer.helper.ts` and `sshGit.helper.ts` used to each probe with their
* own local `spawnSync` check and let a missing dependency silently skip the
* spec, in CI as well as locally. These wrappers keep that local-dev
* behavior but throw under CI, where the dependency is expected to be
* present and a skip would be a false claim of coverage.
*/
import { spawnSync } from 'child_process';
export type DependencyProbe = () => boolean;
export const defaultGitProbe: DependencyProbe = () =>
spawnSync('git', ['--version'], { stdio: 'ignore' }).status === 0;
export const defaultSshdProbe: DependencyProbe = () =>
spawnSync('/usr/sbin/sshd', ['-V'], { stdio: 'ignore' }).status === 0;
function requireDependency(name: string, hint: string, probe: DependencyProbe): boolean {
const present = probe();
if (!present && process.env.CI) {
throw new Error(`${name} is required in CI but was not found. ${hint}`);
}
return present;
}
/** True when the system `git` binary is available; throws under CI if not. */
export function requireGitBinary(probe: DependencyProbe = defaultGitProbe): boolean {
return requireDependency(
'git',
'Ensure the CI image installs the git CLI before running the E2E suite.',
probe,
);
}
/** True when a local `sshd` binary is available; throws under CI if not. */
export function requireSshd(probe: DependencyProbe = defaultSshdProbe): boolean {
return requireDependency(
'sshd',
'Ensure the CI image installs openssh-server and frees loopback port 22 (see .github/workflows/ci.yml).',
probe,
);
}
+4 -3
View File
@@ -12,17 +12,18 @@
* NODE_EXTRA_CA_CERTS (wired in CI and in the local validation lifecycle).
* The key is a throwaway test certificate with no security value.
*
* Soft-skips when the system git binary is unavailable.
* Soft-skips when the system git binary is unavailable (locally; throws
* under CI, see externalDeps.ts).
*/
import { spawn, spawnSync } from 'child_process';
import fs from 'fs';
import https from 'https';
import os from 'os';
import path from 'path';
import { requireGitBinary } from './externalDeps';
export function gitAvailable(): boolean {
const probe = spawnSync('git', ['--version'], { stdio: 'ignore' });
return probe.status === 0;
return requireGitBinary();
}
/** Build a git repository with the given files on `branch`, returns the repo dir. */
+2 -2
View File
@@ -10,10 +10,10 @@ import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'fs';
import net from 'net';
import os from 'os';
import path from 'path';
import { requireGitBinary, requireSshd } from './externalDeps';
export function sshGitFixtureAvailable(): boolean {
return spawnSync('git', ['--version'], { stdio: 'ignore' }).status === 0
&& spawnSync('/usr/sbin/sshd', ['-V'], { stdio: 'ignore' }).status === 0;
return requireGitBinary() && requireSshd();
}
const COMPOSE_FIXTURE = `services: