Files
sencho/backend/src/__tests__/__helpers__/testHandleResolver.ts
T
Anso 0928765232 docs(git): publish a versioned Git transport support matrix (#1883)
* fix(git): make gitSourceStatus exhaustive over GitSourceErrorCode

GIT_ERROR was the only code falling through the implicit default
branch. Give it an explicit case and add the same never-guard
webhookPullStatus already uses, so a future code with no mapping is a
compile error instead of a silent 400.

* fix(git): fail loudly under CI when git or sshd is missing

Every real-git and real-sshd integration suite carried its own local
gitAvailable()/sshdAvailable() probe and skipped silently when the
dependency was absent, in CI as well as locally. A cell in the
upcoming support matrix could then advertise automated proof while
the test that proves it never ran.

Consolidate into shared requireGitBinary()/requireSshd() helpers
(one for backend vitest, one for Playwright, since backend's rootDir
pin blocks a cross-directory import) that take an injectable probe.
Locally a missing dependency still skips; under CI it throws with an
actionable message naming what's missing.

* feat(docs): publish a versioned Git transport support matrix

Adds docs/git-transport-support.yaml as the canonical claim set for
every transport/ref/auth/host/CA combination Git Sources supports,
each claim naming its own reproducible evidence rather than
generalizing from a related test. A claim is supported only when a
real end-to-end test (or a dated live attestation) proves that exact
combination; everything else is marked unverified, never assumed.

The published page (docs/features/git-transport-support.mdx) is
generated from the YAML by backend/scripts/git-support-matrix, so it
cannot silently drift from what the tests actually prove. A new
backend test (git-support-matrix.test.ts) enforces this: schema
validity, evidence semantics (supported needs success evidence,
unsupported needs a reproducible rejection, unverified forbids
evidence entirely), byte-identical page generation, and that every
referenced test title resolves via the TypeScript AST rather than a
string search that a skipped or commented-out test would pass.

The error-model section is cross-checked against the real
GitSourceErrorCode and TransportFacingCode unions and against
gitSourceStatus's actual HTTP mapping, so the matrix and the runtime
behavior cannot diverge either.

Named Git hosts (GitHub, GitLab, Gitea, Forgejo, Bitbucket) and the
direct-proxy/Pilot execution paths are seeded as unverified pending a
live attestation pass; only the generic local-fixture combinations
already proven by the real-git integration suites are marked
supported today.

* docs(git): scope GitHub claims to what this pass can actually attest

Splits the GitHub row into a public no-auth claim (attestable with a
real public repository) and separate PAT/SSH deploy-key claims marked
unverified with an explicit reason: this pass holds no real GitHub
credential to exercise them with, and none is assumed or fabricated.

* feat(docs): attest the Git transport matrix live against real hosts

Runs the QA fleet's live Sencho instance through the transport
combinations that automated fixtures cannot exercise, then records
each result in docs/git-transport-attestations.yaml so it can be
re-run and compared later.

GitHub, GitLab, and Bitbucket are attested over public HTTPS against
real, stable, publicly-owned demo repositories (branch and pinned
SHA; GitLab additionally has a tagged fixture). Gitea and Forgejo get
full coverage (branch, tag, and SHA, over both HTTPS with a
per-source CA and SSH with a deploy key) against disposable
self-hosted instances stood up for this pass, including a private
repository so the authentication and host-key failure classifiers
were exercised against a real wrong credential and a real wrong host
key, not just the mocked corpus. The direct-proxy and Pilot execution
paths are each confirmed once against a real public host, proving
the distributed dispatch itself rather than assuming it from the
local-path evidence.

Left honestly unverified: GitHub PAT and SSH deploy-key auth (this
pass holds no real GitHub credential), a GitHub tag combination (no
small stable tagged fixture found), and a Bitbucket tag combination
(the fixture repository carries none). Every claim's evidence records
its exact transport, ref, auth, host, CA, and node path so nothing
here is extrapolated from a neighboring result.

All infrastructure created for this pass (two throwaway Git server
containers, one probe stack) was torn down afterward and the fleet's
container list was confirmed to match its state before the pass.

* style(git): replace em dashes and fix a stale .mjs reference

Directive 18 applies to code comments and build markers too, not just
prose. Also corrects the claim set's header comment, which still
named render.mjs after the renderer was moved to render.js to match
the house convention for backend scripts.
2026-09-01 21:56:02 -04:00

151 lines
6.5 KiB
TypeScript

/**
* Resolves a (file, exact test title) proof handle against real vitest
* source, using the TypeScript compiler API rather than a string search.
*
* A string search accepts a commented-out test, a `.skip`-ed test, or a
* duplicate title landing on the wrong declaration. This walks the actual
* AST: it finds every `it`/`test` declaration with that literal title,
* rejects any declaration carrying a skip-shaped modifier (`.skip`, `.todo`,
* `.failing`, `.only`, `.each`, `.skipIf`, `.runIf`) directly, and rejects
* any declaration nested under an enclosing `describe` that is unconditionally
* skipped or conditionally skipped by anything other than
* `describe.skipIf(...)` whose predicate calls one of the approved hardened
* dependency probes (`requireGitBinary`, `requireSshd` from
* `./externalDeps`). `describe.skip` and `describe.runIf` are never
* approved, at any nesting depth.
*/
import ts from 'typescript';
export const APPROVED_GUARD_HELPERS = ['requireGitBinary', 'requireSshd'];
const SKIP_SHAPED_MODIFIERS = new Set(['skip', 'todo', 'failing', 'only', 'each', 'skipIf', 'runIf']);
export type HandleResolution =
| { ok: true }
| { ok: false; reason: 'not-found' | 'duplicate' | 'skipped-directly' | 'unapproved-ancestor-skip' };
interface CallShape {
kind: string;
modifier: string | null;
/** For a curried modifier (`X.skipIf(pred)(title, fn)`), the inner call's first argument. */
predicateArg?: ts.Node;
}
const CURRIED_MODIFIERS = new Set(['skipIf', 'runIf', 'each']);
function classifyCall(node: ts.CallExpression): CallShape | null {
const expr = node.expression;
// Direct form: describe('x', fn) / describe.skip('x', fn) / it.todo('x').
if (ts.isIdentifier(expr)) {
return { kind: expr.text, modifier: null };
}
if (ts.isPropertyAccessExpression(expr) && ts.isIdentifier(expr.expression)) {
return { kind: expr.expression.text, modifier: expr.name.text };
}
// Curried form: X.skipIf(pred)(title, fn) / X.runIf(pred)(title, fn) /
// X.each(cases)(title, fn): the outer call's expression is itself a
// CallExpression whose own expression is the X.modifier access.
if (ts.isCallExpression(expr) && ts.isPropertyAccessExpression(expr.expression) && ts.isIdentifier(expr.expression.expression)) {
const modifier = expr.expression.name.text;
if (CURRIED_MODIFIERS.has(modifier)) {
return { kind: expr.expression.expression.text, modifier, predicateArg: expr.arguments[0] };
}
}
return null;
}
function predicateCallsApprovedHelper(argNode: ts.Node): boolean {
let found = false;
const visit = (n: ts.Node): void => {
if (ts.isCallExpression(n) && ts.isIdentifier(n.expression) && APPROVED_GUARD_HELPERS.includes(n.expression.text)) {
found = true;
}
ts.forEachChild(n, visit);
};
visit(argNode);
return found;
}
function stringLiteralText(node: ts.Node | undefined): string | null {
if (node && ts.isStringLiteralLike(node)) return node.text;
return null;
}
function functionBodyArgument(node: ts.CallExpression): ts.ArrowFunction | ts.FunctionExpression | undefined {
return node.arguments.find((a): a is ts.ArrowFunction | ts.FunctionExpression =>
ts.isArrowFunction(a) || ts.isFunctionExpression(a));
}
type AncestorState = 'none' | 'approved' | 'unapproved';
interface FoundDeclaration {
title: string;
ownModifier: string | null;
ancestorState: AncestorState;
}
function collectDeclarations(sourceFile: ts.SourceFile): FoundDeclaration[] {
const found: FoundDeclaration[] = [];
function walk(node: ts.Node, ancestorState: AncestorState): void {
if (ts.isCallExpression(node)) {
const classified = classifyCall(node);
if (classified?.kind === 'describe') {
let nextState: AncestorState = ancestorState;
if (ancestorState !== 'unapproved') {
if (classified.modifier === 'skip' || classified.modifier === 'runIf') {
nextState = 'unapproved';
} else if (classified.modifier === 'skipIf') {
const predicate = classified.predicateArg;
nextState = predicate && predicateCallsApprovedHelper(predicate) ? 'approved' : 'unapproved';
} else if (classified.modifier === 'only' || classified.modifier === 'each' || classified.modifier === 'todo') {
// Scoping/parameterization modifiers on describe don't skip
// this suite's tests; leave ancestorState unchanged.
nextState = ancestorState;
}
}
const callback = functionBodyArgument(node);
if (callback?.body) {
ts.forEachChild(callback.body, (child) => walk(child, nextState));
}
return;
}
if (classified?.kind === 'it' || classified?.kind === 'test') {
const title = stringLiteralText(node.arguments[0]);
if (title !== null) {
found.push({ title, ownModifier: classified.modifier, ancestorState });
}
// Do not descend further into an it/test call's own arguments;
// its callback body is the test implementation, not more
// declarations.
return;
}
}
ts.forEachChild(node, (child) => walk(child, ancestorState));
}
walk(sourceFile, 'none');
return found;
}
export function resolveTestHandle(filePath: string, sourceText: string, title: string): HandleResolution {
const sourceFile = ts.createSourceFile(filePath, sourceText, ts.ScriptTarget.Latest, true);
const declarations = collectDeclarations(sourceFile).filter((d) => d.title === title);
if (declarations.length === 0) return { ok: false, reason: 'not-found' };
const runnable = declarations.filter((d) => d.ownModifier === null && d.ancestorState !== 'unapproved');
if (runnable.length > 1) return { ok: false, reason: 'duplicate' };
if (runnable.length === 1) return { ok: true };
// Every matching declaration is skipped some way; report the most
// specific reason from the first match.
const first = declarations[0];
if (first.ownModifier !== null && SKIP_SHAPED_MODIFIERS.has(first.ownModifier)) {
return { ok: false, reason: 'skipped-directly' };
}
return { ok: false, reason: 'unapproved-ancestor-skip' };
}