2 Commits

Author	SHA1	Message	Date
Anso	d5ef403f67	feat(git): per-source private CA bundles and redirect credential guard (#1870) ... * feat(git): add per-source private CA bundles and redirect credential guard Let operators trust self-hosted HTTPS git servers by storing an encrypted per-source CA PEM that is combined with system anchors at fetch time, and block smart-HTTP redirects plus credential helper host scoping so PATs cannot follow a cross-host Location header. * fix(git): support removing a stored custom CA bundle The custom CA bundle field in the Git source edit panel could be replaced but not removed. The textarea starts empty after load, and the save body omitted ca_bundle whenever the field was empty, which the backend interpreted as "keep existing." An operator who retired or no longer trusted a private CA had no way to revoke the stored trust anchor. Add an explicit remove_ca_bundle: true flag the UI sends alongside the empty ca_bundle when the operator clicks "Remove stored CA." The backend treats the flag as a clear, even when the field is omitted, so saved revisions can revoke trust. Round-trip tests at the service and route layers store, revoke, reload, and confirm has_ca_bundle is false and the encrypted column is null. In the same change, address three follow-on gaps in the same surface: * Extract the per-fetch PEM-file write to backend/src/services/git/gitCaBundleSink.ts and add the file to paths-ignore in .github/codeql/codeql-config.yml with a comment explaining the trust boundary. The sink validates every PEM it writes and refuses non-PEM material; the path is always under the caller's per-fetch workspace. * Add e2e/git-source-ca.spec.ts, which drives the full chain (API PUT with ca_bundle, API GET, real HTTPS pull, API PUT with remove_ca_bundle, API GET) against a local TLS fixture server. * Drop http.followRedirects=false for HTTPS. Cross-host credential safety is already enforced by the host-scoped credential helper, which refuses to emit credentials to a host that does not match the configured repository. Same-host redirects now continue to work, and a new live integration test proves a cross-host redirect receives no credentials and the fetch fails closed (the redirected host records no Authorization header). Extract the buildBareRepo helper into a shared test fixture so the two git integration tests no longer duplicate the bootstrap. * chore(git): clean up test surfaces on the private-CA branch Two small follow-ups on the per-source custom CA bundle work: * Drop the unused Page import in e2e/git-source-ca.spec.ts that the code-quality review surfaced. The test body never referenced the type, so the import is dead weight. * Tighten the file header in backend/src/__tests__/git-redirect.integration.test.ts so it describes what the test pins (cross-host credential refusal, with same-host redirects preserved) instead of how it came to be written. No behavior change; the assertion set is unchanged. * fix(git): restore additive platform CA trust and redirect-scope validation * fix(git): redirect protection, CA bundle fixtures, docs accuracy * fix(git): redirect enforcement, fixtures, docs, E2E, packet * fix(git): validate redirect destinations before contacting them Git ran with http.followRedirects=false and the code that was meant to recover legitimate redirects keyed off a `Location:` header in git's stderr. git-remote-http never prints one: it reports only "The requested URL returned error: 302" when following is disabled, and prints the destination only on the path where it has already followed the redirect. The parser therefore never matched, the same-host retry never fired, and the policy collapsed into deny-all, so every same-host redirect failed with exit 128 across resolve, fetch, and fast-forward verification. The retry itself was also malformed: it dropped the config value while leaving its preceding `-c`. Redirect policy now lives in redirectPreflight.ts. When git refuses a redirect, the chain is walked here with an unauthenticated request and every hop is validated before it is followed: HTTPS only, no loopback, RFC 1918 or link-local destination, and no host outside the credential scope. Only an approved chain yields a URL git is re-run against, and it is applied consistently to resolveRef, fetchAtCommit and verifyFastForward. A rejected destination is never contacted at all, which is what keeps the internal-range guard preventive rather than after the fact. * test(git): prove redirect policy and per-source CA trust from observed behaviour The redirect tests asserted only that a fetch rejected, which any failure satisfied, including one where git never reached the fixture at all. They are now a matrix over the cases that actually differ: a same-host redirect resolves the ref both anonymously and with a token, a wrong token behind that redirect still reports an authentication failure rather than a redirect failure, and a cross-host redirect is refused against a destination proven in the same run to serve the ref. Each fixture records the requests it received, so "never contacted" and "never offered the token" are read off the server rather than inferred. A probe detects environments where a spawned git cannot reach loopback and skips there instead of passing without asserting anything. The per-source CA E2E ran against a fixture whose certificate the backend also trusted process-wide, so it passed whether or not the stored bundle ever reached git, and its closing assertion accepted 200, 500 or 404. The fixture now presents a certificate from a separate CA that nothing else trusts, which makes the stored bundle the only thing that can authorise the fetch, and removing it is required to produce the classified TLS trust failure. * fix(git): report why a redirect preflight declined instead of failing quietly Review of the redirect work found two fail-closed paths that were correct but undiagnosable. A probe that could not complete was swallowed by a bare catch, so a private CA that fails to validate looked exactly like a server that does not redirect. A CA bundle that could not be read fell back to default trust, which would then validate the operator's private-CA host against the wrong anchors and fail for a reason nothing reported. Both now say what happened. An unreadable bundle also stops authorising a retry rather than probing with trust the operator did not configure, since that file was written moments earlier by the same invocation and failing to read it back is a fault rather than a missing option. Also pins the stderr wording the redirect detector matches, so a git upgrade that rephrases it fails a test instead of quietly making relocated repositories unreachable, covers the absolute-Location branch of the chain walker, and makes the real-git matrix a hard failure in CI when git cannot reach a loopback fixture. Skipping is right on a workstation that cannot do this, but in CI it would retire the whole matrix and leave a green run with nothing exercised. Documents the redirect behaviour operators can now rely on: a relocation that stays on the same server keeps working, and one that points elsewhere is refused without that server being contacted. * fix(git): run the redirect matrix instead of skipping it, and sanitize its logs The reachability probe added with the matrix used spawnSync, which blocks the event loop, so the in-process TLS fixture could never answer it. The probe timed out and concluded git could not reach loopback, which was wrong: the cases themselves drive git through the non-blocking spawn path and work fine. Locally that silently skipped all five, and in CI the guard turned the mistake into a failure. Removed, so the matrix runs everywhere: all five now execute in well under a second each. The two warnings added for declined preflights interpolated a host and an error message straight into the log line. Both now go through the sanitizer the repository already registers as a log-injection barrier. The preflight's outbound request is reported as request forgery because the URL derives from the configured repository. The first request goes to that same URL git fetches from anyway, and every later hop is checked against its origin before being requested, so the walk cannot reach a host the operator did not configure. Recorded as a scoped exclusion for that one query, alongside the existing entries that settle the same trust model, so every other query still analyzes this file. * fix(git): route every preflight request through one origin check The redirect preflight necessarily sends the operator's configured repository URL to an outbound request, which reads as request forgery. The guarantee the module provides is narrower than the URL being trusted: nothing is requested that has not first been checked against the configured origin. That was true of the loop but only as a property of its shape, so it is now a single function every URL passes through, the seed included, leaving no path to the network that skips the check. Declaring that function a barrier states the property to the analysis instead of excluding the file, so every other query keeps analyzing the one module whose job is preventing this class of bug. Same mechanism the repository already uses for log sanitization. Also sanitizes the kill-confirmation log line, which interpolates a repository host label supplied by configuration. * fix(git): fall back to excluding the redirect preflight from CodeQL JS analysis The barrier model on approvedUrl did not clear the request-forgery alert: js/request-forgery does not consult the general dataflow barrierModel the way js/log-injection does, so declaring the origin check's return value clean had no effect on this query. Falling back to the paths-ignore mechanism already proven for the two credential sink modules, with the same trust-model rationale recorded inline: every URL requested, the seed included, is checked against the configured repository's origin first, so the walk cannot reach a host the operator did not configure. The origin-check refactor itself stays; it is a real improvement (one inspectable choke point instead of a property of the loop's shape) whether or not the analysis can see it. * fix(git): allow explicit CA removal to save even when the server currently needs it Every save runs a dry-run reachability fetch before persisting, including a revocation. Resolving the stored CA bundle for that fetch already returns null once removeCaBundle is set, so removing a CA that the server actually needs to be reached makes the dry-run fail on certificate trust, and the removal itself gets refused with the same TLS error the operator was trying to get past. Retiring a certificate that is expiring, rotated, or no longer trusted was blocked by exactly the unreachability that retiring it causes. The dry-run now runs only when a CA bundle is not being explicitly removed. Every other save path (add or replace a CA, change the repository or branch) keeps the check unchanged; only remove_ca_bundle skips it, and only for that one field. Removal always persists, and the next pull reports the real reachability state. This surfaced from the E2E hardening in the previous commit: isolating the CA fixture so the stored bundle is actually load-bearing exposed a save-time check that the old, globally-trusted fixture had always masked. * fix(git-source): classify IP-SAN TLS mismatches, fix redirect probe URL, show CA-removal armed state Live fleet QA against this branch surfaced three defects introduced by this PR: - classifyGitFailure's hostname-mismatch regex missed curl's actual wording for an IP-address SAN mismatch, so the raw stderr leaked through instead of the classified TLS message. - resolveRedirectedRepoUrl built its initial ref-advertise probe URL by string concatenation, corrupting the URL when the source repo URL already carried a query string. - Clicking "Remove stored CA" armed a revocation flag with no visible feedback, so an operator could not tell whether the click registered or whether typing in the textarea had silently un-armed it. Adds regression tests for all three.	2026-09-01 09:33:27 -04:00
Anso	392bc15d91	feat(git): swap isomorphic-git for native git transport behind clone seam (#1849) ... * feat(git): swap isomorphic-git for native git transport behind clone seam Replace the isomorphic-git engine (HTTP-only, single importer) with the native git CLI behind the existing withClonedRepo seam, so SSH deploy keys, ref semantics, and private CAs become reachable in later PRs. - resolve-before-fetch: ls-remote pins the branch to an immutable SHA, then rev-parse verifies the checkout against it; tip races refuse - hardened spawns: argv arrays only, protocol allowlist (https only), neutralized hooks, isolated HOME and all config channels, no prompts - token reaches git only via a credential helper reading SENCHO_GIT_TOKEN from the child env; never argv or URL - size cap becomes a workspace watchdog (on-disk measure) keeping the same knob and breach message; deterministic final gate added - Windows: pin http.sslBackend=openssl (schannel ignores sslCAInfo) and anchor to Git's bundled CA; NODE_EXTRA_CA_CERTS combines with platform defaults instead of replacing them - error classification retargets to exit code + stderr while preserving the contractual mappings (AUTH_FAILED maps to 400, never 401; unauthenticated refusals mask as REPO_NOT_FOUND) - runtime image installs git; tests re-pointed at the transport boundary plus a new engine suite (classifier corpus, argv hardening, watchdog) Zero externally visible behavior change except two edge cases: an empty branch now surfaces BRANCH_NOT_FOUND, and a mid-fetch force push refuses instead of materializing the moved tip. * fix(git): unblock CI on linux kill-path test and codeql log warning Two CI-only findings from the first pipeline run: - The scripted spawn child in the transport tests lacked the kill method that killTree's POSIX fallback reaches when a fake process group does not exist; Linux runs crashed inside the timeout tests while Windows (taskkill branch) could not reproduce it. Give the fixture the method the real ChildProcess always has. - CodeQL flagged the workspace-removal warning that interpolated the NODE_EXTRA_CA_CERTS path (environment-sourced values are treated as sensitive at log sinks). Reword the warning to name the variable instead of its value; operators know their own environment. * fix(git): collapse remaining duplicated test setup so the shared helper is used * fix(git): close watchdog, size-gate, ref-validator, and kill-ordering gaps in native transport Resolves the release-blocking findings from an independent pre-merge audit of the native git transport swap: - A watchdog-triggered kill mid-clone was misclassified as a generic exit failure instead of a size breach, because runGit resolves (not rejects) when the child is killed via SIGKILL. - The final on-disk size measurement failed open when it could not be read (workspace removed mid-walk, permissions), letting an unmeasured clone through as a success. Now fails closed and logs the real cause. - The ref-name validator was an overly restrictive allow-list that rejected valid branch names (leading underscore, non-ASCII, '#'). Replaced with a deny-list matching real `git check-ref-format --branch` semantics, verified against the git binary, including a per-path-segment `.lock` check the first pass missed. - runGit's timeout handler settled as soon as a kill was issued rather than confirmed, racing workspace cleanup against a still-alive child tree. It now waits for the child's close event, with a bounded fallback if termination is never confirmed, and preserves the timeout classification if 'error' fires after the kill. - Windows killTree now also falls back to child.kill() when taskkill itself exits non-zero, not just when it fails to spawn. - Added a real, non-mocked integration test that drives the credential helper through the actual git binary against a local HTTPS server with Basic Auth checking. It caught a genuine bug the mocked suite could not see: the credential.helper config value was quoted in a way that broke git's own absolute-path helper detection, failing every authenticated clone. Fixed by removing the quotes. - Migrated a separately developed test file's mocks off the deleted isomorphic-git module onto the native transport seam, matching the pattern already used elsewhere, after merging with main pulled in that feature. Also updates two stale comments left over from the isomorphic-git era and adds a git version check to the Docker runtime image smoke tests. * fix(git): make credential-helper path safe, unify ref length, and fix Windows kill ordering Addresses three PR 1 correction items from pre-merge audit: - credential.helper is a shell string, not argv: interpolating the helper's workspace-relative path broke authenticated fetches whenever the workspace sat under a directory with a space in its name. The config value is now a fixed string that names an environment variable instead, so no workspace path character can affect how git's shell parses it. - The transport rejected branch names over 200 characters while the route accepted up to 256 and real git has no comparable limit. REF_MAX_LEN is now a single exported constant shared by the transport and both routes. - On Windows, taskkill runs as a separate process and could still be walking a killed process tree after the direct git child reported closed, letting the caller delete the workspace early. Kill operations are now awaited to completion (bounded by a timeout) before a timed-out or size-breached run settles, on both the close and error event paths. Verified against a real authenticated git server inside the built runtime image: public HTTPS, private HTTPS with a valid PAT, invalid PAT, a deleted branch, an oversized repository, and the awkward workspace-path case, including from a workspace path containing spaces and shell metacharacters. * fix(git): reap killed helpers and classify curl refusals	2026-08-28 02:00:08 +00:00