mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-28 11:17:07 +00:00
feat(git): swap isomorphic-git for native git transport behind clone seam (#1849)
* feat(git): swap isomorphic-git for native git transport behind clone seam Replace the isomorphic-git engine (HTTP-only, single importer) with the native git CLI behind the existing withClonedRepo seam, so SSH deploy keys, ref semantics, and private CAs become reachable in later PRs. - resolve-before-fetch: ls-remote pins the branch to an immutable SHA, then rev-parse verifies the checkout against it; tip races refuse - hardened spawns: argv arrays only, protocol allowlist (https only), neutralized hooks, isolated HOME and all config channels, no prompts - token reaches git only via a credential helper reading SENCHO_GIT_TOKEN from the child env; never argv or URL - size cap becomes a workspace watchdog (on-disk measure) keeping the same knob and breach message; deterministic final gate added - Windows: pin http.sslBackend=openssl (schannel ignores sslCAInfo) and anchor to Git's bundled CA; NODE_EXTRA_CA_CERTS combines with platform defaults instead of replacing them - error classification retargets to exit code + stderr while preserving the contractual mappings (AUTH_FAILED maps to 400, never 401; unauthenticated refusals mask as REPO_NOT_FOUND) - runtime image installs git; tests re-pointed at the transport boundary plus a new engine suite (classifier corpus, argv hardening, watchdog) Zero externally visible behavior change except two edge cases: an empty branch now surfaces BRANCH_NOT_FOUND, and a mid-fetch force push refuses instead of materializing the moved tip. * fix(git): unblock CI on linux kill-path test and codeql log warning Two CI-only findings from the first pipeline run: - The scripted spawn child in the transport tests lacked the kill method that killTree's POSIX fallback reaches when a fake process group does not exist; Linux runs crashed inside the timeout tests while Windows (taskkill branch) could not reproduce it. Give the fixture the method the real ChildProcess always has. - CodeQL flagged the workspace-removal warning that interpolated the NODE_EXTRA_CA_CERTS path (environment-sourced values are treated as sensitive at log sinks). Reword the warning to name the variable instead of its value; operators know their own environment. * fix(git): collapse remaining duplicated test setup so the shared helper is used * fix(git): close watchdog, size-gate, ref-validator, and kill-ordering gaps in native transport Resolves the release-blocking findings from an independent pre-merge audit of the native git transport swap: - A watchdog-triggered kill mid-clone was misclassified as a generic exit failure instead of a size breach, because runGit resolves (not rejects) when the child is killed via SIGKILL. - The final on-disk size measurement failed open when it could not be read (workspace removed mid-walk, permissions), letting an unmeasured clone through as a success. Now fails closed and logs the real cause. - The ref-name validator was an overly restrictive allow-list that rejected valid branch names (leading underscore, non-ASCII, '#'). Replaced with a deny-list matching real `git check-ref-format --branch` semantics, verified against the git binary, including a per-path-segment `.lock` check the first pass missed. - runGit's timeout handler settled as soon as a kill was issued rather than confirmed, racing workspace cleanup against a still-alive child tree. It now waits for the child's close event, with a bounded fallback if termination is never confirmed, and preserves the timeout classification if 'error' fires after the kill. - Windows killTree now also falls back to child.kill() when taskkill itself exits non-zero, not just when it fails to spawn. - Added a real, non-mocked integration test that drives the credential helper through the actual git binary against a local HTTPS server with Basic Auth checking. It caught a genuine bug the mocked suite could not see: the credential.helper config value was quoted in a way that broke git's own absolute-path helper detection, failing every authenticated clone. Fixed by removing the quotes. - Migrated a separately developed test file's mocks off the deleted isomorphic-git module onto the native transport seam, matching the pattern already used elsewhere, after merging with main pulled in that feature. Also updates two stale comments left over from the isomorphic-git era and adds a git version check to the Docker runtime image smoke tests. * fix(git): make credential-helper path safe, unify ref length, and fix Windows kill ordering Addresses three PR 1 correction items from pre-merge audit: - credential.helper is a shell string, not argv: interpolating the helper's workspace-relative path broke authenticated fetches whenever the workspace sat under a directory with a space in its name. The config value is now a fixed string that names an environment variable instead, so no workspace path character can affect how git's shell parses it. - The transport rejected branch names over 200 characters while the route accepted up to 256 and real git has no comparable limit. REF_MAX_LEN is now a single exported constant shared by the transport and both routes. - On Windows, taskkill runs as a separate process and could still be walking a killed process tree after the direct git child reported closed, letting the caller delete the workspace early. Kill operations are now awaited to completion (bounded by a timeout) before a timed-out or size-breached run settles, on both the close and error event paths. Verified against a real authenticated git server inside the built runtime image: public HTTPS, private HTTPS with a valid PAT, invalid PAT, a deleted branch, an oversized repository, and the awkward workspace-path case, including from a workspace path containing spaces and shell metacharacters. * fix(git): reap killed helpers and classify curl refusals
This commit is contained in:
Anso
@@ -171,7 +171,7 @@ jobs:
|
||||
run: |
|
||||
set -euo pipefail
|
||||
docker run --rm --entrypoint sh sencho:pr-test -c \
|
||||
'docker --version && docker compose version'
|
||||
'docker --version && docker compose version && git --version'
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# E2E Tests (PRs only, skipped for release-please PRs)
|
||||
|
||||
@@ -133,7 +133,7 @@ jobs:
|
||||
run: |
|
||||
set -euo pipefail
|
||||
docker run --rm --entrypoint sh localhost/sencho:dev-scan -c \
|
||||
'docker --version && docker compose version'
|
||||
'docker --version && docker compose version && git --version'
|
||||
|
||||
docker run -d --name sencho-dev-smoke -p 1852:1852 localhost/sencho:dev-scan
|
||||
trap 'docker logs sencho-dev-smoke 2>&1 || true; docker rm -f sencho-dev-smoke >/dev/null 2>&1 || true' EXIT
|
||||
|
||||
@@ -163,7 +163,7 @@ jobs:
|
||||
run: |
|
||||
set -euo pipefail
|
||||
docker run --rm --entrypoint sh localhost/sencho:preview-scan -c \
|
||||
'docker --version && docker compose version'
|
||||
'docker --version && docker compose version && git --version'
|
||||
|
||||
docker run -d --name sencho-preview-smoke -p 1852:1852 localhost/sencho:preview-scan
|
||||
trap 'docker logs sencho-preview-smoke 2>&1 || true; docker rm -f sencho-preview-smoke >/dev/null 2>&1 || true' EXIT
|
||||
|
||||
@@ -153,10 +153,10 @@ jobs:
|
||||
- name: Smoke test release image (pre-publish)
|
||||
run: |
|
||||
set -euo pipefail
|
||||
# Verify source-built Docker CLI and Compose binaries are present and functional.
|
||||
# Both checks run in one container to avoid double start-up overhead.
|
||||
# Verify source-built Docker CLI, Compose, and git binaries are present and functional.
|
||||
# These checks run in one container to avoid repeated start-up overhead.
|
||||
docker run --rm --entrypoint sh localhost/sencho:release-scan -c \
|
||||
'docker --version && docker compose version'
|
||||
'docker --version && docker compose version && git --version'
|
||||
|
||||
# Not using --rm so that a crashed container sticks around long
|
||||
# enough for `docker logs` in the trap to surface the stack trace.
|
||||
|
||||
+7
-6
@@ -275,13 +275,14 @@ FROM node:26-alpine@sha256:aadf416b2cdce311a8811ba3f0608a61b77dbf997500e2eafe781
|
||||
ARG APK_CACHE_BUST=unset
|
||||
|
||||
# Upgrade all Alpine system packages and install runtime deps.
|
||||
# Docker CLI and Compose are copied from source-built stages below,
|
||||
# git is required at runtime: Git Sources clones through the native git
|
||||
# client. Docker CLI and Compose are copied from source-built stages below,
|
||||
# eliminating the curl dependency and all Go stdlib CVEs from the upstream
|
||||
# static binaries. npm is removed because it is not needed at runtime;
|
||||
# removing it also eliminates CVE-2026-33671 (picomatch ReDoS in npm).
|
||||
RUN echo "apk cache bust: ${APK_CACHE_BUST}" && \
|
||||
apk upgrade --no-cache && \
|
||||
apk add --no-cache bash su-exec && \
|
||||
apk add --no-cache bash su-exec git tini && \
|
||||
mkdir -p /usr/local/lib/docker/cli-plugins
|
||||
|
||||
# Copy the source-built Docker CLI and Compose plugin from their builder stages.
|
||||
@@ -349,8 +350,8 @@ EXPOSE 1852
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
|
||||
CMD node -e "const h=require('http');h.get('http://localhost:1852/api/health',r=>{process.exit(r.statusCode===200?0:1)}).on('error',()=>process.exit(1))"
|
||||
|
||||
# Entrypoint ensures /app/data is writable and execs the CMD as root by default,
|
||||
# or drops to $SENCHO_USER via su-exec when that env var is set (see comment above).
|
||||
# CMD provides the default arguments passed through to the entrypoint.
|
||||
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
|
||||
# Tini owns PID 1 so orphaned Git transport helpers are reaped after a
|
||||
# process-group kill. The entrypoint still prepares /app/data before execing
|
||||
# the application command.
|
||||
ENTRYPOINT ["/sbin/tini", "--", "/usr/local/bin/docker-entrypoint.sh"]
|
||||
CMD ["node", "dist/index.js"]
|
||||
|
||||
Generated
+2
-446
@@ -10,7 +10,6 @@
|
||||
"hasInstallScript": true,
|
||||
"license": "AGPL-3.0-only",
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "3.1111.0",
|
||||
"axios": "^1.15.0",
|
||||
"bcrypt": "^6.0.0",
|
||||
"better-sqlite3": "^13.0.1",
|
||||
@@ -25,7 +24,6 @@
|
||||
"helmet": "^8.1.0",
|
||||
"http-proxy": "^1.18.1",
|
||||
"http-proxy-middleware": "^4.0.0",
|
||||
"isomorphic-git": "^1.37.5",
|
||||
"jsonwebtoken": "^9.0.3",
|
||||
"ldapts": "^9.0.0",
|
||||
"multer": "^2.1.1",
|
||||
@@ -2028,18 +2026,6 @@
|
||||
"url": "https://opencollective.com/vitest"
|
||||
}
|
||||
},
|
||||
"node_modules/abort-controller": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz",
|
||||
"integrity": "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"event-target-shim": "^5.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6.5"
|
||||
}
|
||||
},
|
||||
"node_modules/accepts": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/accepts/-/accepts-2.0.0.tgz",
|
||||
@@ -2212,33 +2198,12 @@
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/async-lock": {
|
||||
"version": "1.4.1",
|
||||
"resolved": "https://registry.npmjs.org/async-lock/-/async-lock-1.4.1.tgz",
|
||||
"integrity": "sha512-Az2ZTpuytrtqENulXwO3GGv1Bztugx6TT37NIo7imr/Qo0gsYiGtSdBa2B6fsXhTpVZDNfu1Qn3pk531e3q+nQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/asynckit": {
|
||||
"version": "0.4.0",
|
||||
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
|
||||
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/available-typed-arrays": {
|
||||
"version": "1.0.7",
|
||||
"resolved": "https://registry.npmjs.org/available-typed-arrays/-/available-typed-arrays-1.0.7.tgz",
|
||||
"integrity": "sha512-wvUjBtSGN7+7SjNpq/9M2Tg350UZD3q62IFZLbRAR1bSMlCo1ZaeW+BJ+D090e4hIIZLBcTDWe4Mh4jvUDajzQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"possible-typed-array-names": "^1.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/axios": {
|
||||
"version": "1.19.0",
|
||||
"resolved": "https://registry.npmjs.org/axios/-/axios-1.19.0.tgz",
|
||||
@@ -2413,6 +2378,7 @@
|
||||
"version": "13.0.3",
|
||||
"resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-13.0.3.tgz",
|
||||
"integrity": "sha512-RbOBxmLBG8uvFUc15X9+9SFemKcQ0WBuISBVkpuiaUB2qblC8UWlHEjdWVoZ8AdhSwmoEgsiXKfopX0CQxaACQ==",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"node-addon-api": "^8.0.0"
|
||||
@@ -2579,24 +2545,6 @@
|
||||
"node": ">= 0.8"
|
||||
}
|
||||
},
|
||||
"node_modules/call-bind": {
|
||||
"version": "1.0.9",
|
||||
"resolved": "https://registry.npmjs.org/call-bind/-/call-bind-1.0.9.tgz",
|
||||
"integrity": "sha512-a/hy+pNsFUTR+Iz8TCJvXudKVLAnz/DyeSUo10I5yvFDQJBFU2s9uqQpoSrJlroHUKoKqzg+epxyP9lqFdzfBQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"call-bind-apply-helpers": "^1.0.2",
|
||||
"es-define-property": "^1.0.1",
|
||||
"get-intrinsic": "^1.3.0",
|
||||
"set-function-length": "^1.2.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/call-bind-apply-helpers": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
|
||||
@@ -2676,12 +2624,6 @@
|
||||
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/clean-git-ref": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/clean-git-ref/-/clean-git-ref-2.0.1.tgz",
|
||||
"integrity": "sha512-bLSptAy2P0s6hU4PzuIMKmMJJSE6gLXGH1cntDu7bWJUksvuM+7ReOK61mozULErYvP6a15rnYl0zFDef+pyPw==",
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/cliui": {
|
||||
"version": "8.0.1",
|
||||
"resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz",
|
||||
@@ -2956,18 +2898,6 @@
|
||||
"node": ">=10.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/crc-32": {
|
||||
"version": "1.2.2",
|
||||
"resolved": "https://registry.npmjs.org/crc-32/-/crc-32-1.2.2.tgz",
|
||||
"integrity": "sha512-ROmzCKrTnOwybPcJApAA6WBWij23HVfGVNKqqrZpuyZOHqK2CwHSvpGuyt/UNNvaIjEd8X5IFGp4Mh+Ie1IHJQ==",
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"crc32": "bin/crc32.njs"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=0.8"
|
||||
}
|
||||
},
|
||||
"node_modules/create-require": {
|
||||
"version": "1.1.1",
|
||||
"resolved": "https://registry.npmjs.org/create-require/-/create-require-1.1.1.tgz",
|
||||
@@ -3028,21 +2958,6 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/decompress-response": {
|
||||
"version": "6.0.0",
|
||||
"resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
|
||||
"integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"mimic-response": "^3.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/deep-is": {
|
||||
"version": "0.1.4",
|
||||
"resolved": "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz",
|
||||
@@ -3059,23 +2974,6 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/define-data-property": {
|
||||
"version": "1.1.4",
|
||||
"resolved": "https://registry.npmjs.org/define-data-property/-/define-data-property-1.1.4.tgz",
|
||||
"integrity": "sha512-rBMvIzlpA8v6E+SJZoo++HAYqsLrkg7MSfIinMPFhmkorw7X+dOXVJQs+QT69zGkzMyfDnIMN2Wid1+NbL3T+A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-define-property": "^1.0.0",
|
||||
"es-errors": "^1.3.0",
|
||||
"gopd": "^1.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/delayed-stream": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
|
||||
@@ -3125,12 +3023,6 @@
|
||||
"node": ">=0.3.1"
|
||||
}
|
||||
},
|
||||
"node_modules/diff3": {
|
||||
"version": "0.0.3",
|
||||
"resolved": "https://registry.npmjs.org/diff3/-/diff3-0.0.3.tgz",
|
||||
"integrity": "sha512-iSq8ngPOt0K53A6eVr4d5Kn6GNrM2nQZtC740pzIriHtn4pOQ2lyzEXQMBeVcWERN0ye7fhBsk9PbLLQOnUx/g==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/docker-modem": {
|
||||
"version": "5.0.7",
|
||||
"resolved": "https://registry.npmjs.org/docker-modem/-/docker-modem-5.0.7.tgz",
|
||||
@@ -3520,30 +3412,12 @@
|
||||
"node": ">= 0.6"
|
||||
}
|
||||
},
|
||||
"node_modules/event-target-shim": {
|
||||
"version": "5.0.1",
|
||||
"resolved": "https://registry.npmjs.org/event-target-shim/-/event-target-shim-5.0.1.tgz",
|
||||
"integrity": "sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/eventemitter3": {
|
||||
"version": "4.0.7",
|
||||
"resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-4.0.7.tgz",
|
||||
"integrity": "sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/events": {
|
||||
"version": "3.3.0",
|
||||
"resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz",
|
||||
"integrity": "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.8.x"
|
||||
}
|
||||
},
|
||||
"node_modules/events-universal": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/events-universal/-/events-universal-1.0.1.tgz",
|
||||
@@ -3778,21 +3652,6 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/for-each": {
|
||||
"version": "0.3.5",
|
||||
"resolved": "https://registry.npmjs.org/for-each/-/for-each-0.3.5.tgz",
|
||||
"integrity": "sha512-dKx12eRCVIzqCxFGplyFKJMPvLEWgmNtUrpTiJIR5u97zEhRG8ySrtboPHZXx7daLxQVrl643cTzbab2tkQjxg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"is-callable": "^1.2.7"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/form-data": {
|
||||
"version": "4.0.6",
|
||||
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.6.tgz",
|
||||
@@ -3977,18 +3836,6 @@
|
||||
"node": ">=4"
|
||||
}
|
||||
},
|
||||
"node_modules/has-property-descriptors": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/has-property-descriptors/-/has-property-descriptors-1.0.2.tgz",
|
||||
"integrity": "sha512-55JNKuIW+vq4Ke1BjOTjM2YctQIvCT7GFzHwmfZPGo5wnrgkid0YQtnAleFSqumZm4az3n2BS+erby5ipJdgrg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"es-define-property": "^1.0.0"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/has-symbols": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
|
||||
@@ -4149,6 +3996,7 @@
|
||||
"version": "5.3.2",
|
||||
"resolved": "https://registry.npmjs.org/ignore/-/ignore-5.3.2.tgz",
|
||||
"integrity": "sha512-hsBTNUqQTDwkWtcdYI2i06Y/nUBEsNEDJKjWdigLvegy8kDuJAS8uRlpkkcQpyEXL0Z/pjDy5HBmMjRCJ2gq+g==",
|
||||
"dev": true,
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 4"
|
||||
@@ -4217,18 +4065,6 @@
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/is-callable": {
|
||||
"version": "1.2.7",
|
||||
"resolved": "https://registry.npmjs.org/is-callable/-/is-callable-1.2.7.tgz",
|
||||
"integrity": "sha512-1BC0BVFhS/p0qtw6enp8e+8OD0UrK0oFLztSjNzhcKA3WDuJxxAPXzPuPtKkjEY9UUoEWlX/8fgKeu2S8i9JTA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/is-extglob": {
|
||||
"version": "2.1.1",
|
||||
"resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz",
|
||||
@@ -4286,27 +4122,6 @@
|
||||
"integrity": "sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/is-typed-array": {
|
||||
"version": "1.1.15",
|
||||
"resolved": "https://registry.npmjs.org/is-typed-array/-/is-typed-array-1.1.15.tgz",
|
||||
"integrity": "sha512-p3EcsicXjit7SaskXHs1hA91QxgTw46Fv6EFKKGS5DRFLD8yKnohjF3hxoju94b/OcMZoQukzpPpBE9uLVKzgQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"which-typed-array": "^1.1.16"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/isarray": {
|
||||
"version": "2.0.5",
|
||||
"resolved": "https://registry.npmjs.org/isarray/-/isarray-2.0.5.tgz",
|
||||
"integrity": "sha512-xHjhDr3cNBK0BzdUJSPXZntQUx/mwMS5Rw4A7lPJ90XGAO6ISP/ePDNuo0vhqOZU+UD5JoodwCAAoZQd3FeAKw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/isexe": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz",
|
||||
@@ -4314,71 +4129,6 @@
|
||||
"dev": true,
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/isomorphic-git": {
|
||||
"version": "1.41.4",
|
||||
"resolved": "https://registry.npmjs.org/isomorphic-git/-/isomorphic-git-1.41.4.tgz",
|
||||
"integrity": "sha512-XfZteQRhteAdzOlKcWAeC+Zkx0ZtAh0yIO0EyURAO+mDcIBg5kzKg4UtnYE7GcmKB6qFZc0zMCXdgod/cQXVXw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"async-lock": "^1.4.1",
|
||||
"clean-git-ref": "^2.0.1",
|
||||
"crc-32": "^1.2.0",
|
||||
"diff3": "0.0.3",
|
||||
"ignore": "^5.1.4",
|
||||
"minimisted": "^2.0.0",
|
||||
"pako": "^1.0.10",
|
||||
"pify": "^4.0.1",
|
||||
"readable-stream": "^4.0.0",
|
||||
"sha.js": "^2.4.12",
|
||||
"simple-get": "^4.0.1"
|
||||
},
|
||||
"bin": {
|
||||
"isogit": "cli.cjs"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=14.17"
|
||||
}
|
||||
},
|
||||
"node_modules/isomorphic-git/node_modules/buffer": {
|
||||
"version": "6.0.3",
|
||||
"resolved": "https://registry.npmjs.org/buffer/-/buffer-6.0.3.tgz",
|
||||
"integrity": "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"base64-js": "^1.3.1",
|
||||
"ieee754": "^1.2.1"
|
||||
}
|
||||
},
|
||||
"node_modules/isomorphic-git/node_modules/readable-stream": {
|
||||
"version": "4.7.0",
|
||||
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-4.7.0.tgz",
|
||||
"integrity": "sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"abort-controller": "^3.0.0",
|
||||
"buffer": "^6.0.3",
|
||||
"events": "^3.3.0",
|
||||
"process": "^0.11.10",
|
||||
"string_decoder": "^1.3.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": "^12.22.0 || ^14.17.0 || >=16.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/jose": {
|
||||
"version": "6.2.9",
|
||||
"resolved": "https://registry.npmjs.org/jose/-/jose-6.2.9.tgz",
|
||||
@@ -4953,18 +4703,6 @@
|
||||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/mimic-response": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
|
||||
"integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/minimatch": {
|
||||
"version": "10.2.6",
|
||||
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.6.tgz",
|
||||
@@ -4981,24 +4719,6 @@
|
||||
"url": "https://github.com/sponsors/isaacs"
|
||||
}
|
||||
},
|
||||
"node_modules/minimist": {
|
||||
"version": "1.2.8",
|
||||
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
|
||||
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/minimisted": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/minimisted/-/minimisted-2.0.1.tgz",
|
||||
"integrity": "sha512-1oPjfuLQa2caorJUM8HV8lGgWCc0qqAO1MNv/k05G4qslmsndV/5WdNZrqCiyqiz3wohia2Ij2B7w2Dr7/IyrA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"minimist": "^1.2.5"
|
||||
}
|
||||
},
|
||||
"node_modules/mkdirp-classic": {
|
||||
"version": "0.5.3",
|
||||
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
|
||||
@@ -5338,12 +5058,6 @@
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/pako": {
|
||||
"version": "1.0.11",
|
||||
"resolved": "https://registry.npmjs.org/pako/-/pako-1.0.11.tgz",
|
||||
"integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==",
|
||||
"license": "(MIT AND Zlib)"
|
||||
},
|
||||
"node_modules/parseurl": {
|
||||
"version": "1.3.3",
|
||||
"resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz",
|
||||
@@ -5409,24 +5123,6 @@
|
||||
"url": "https://github.com/sponsors/jonschlinkert"
|
||||
}
|
||||
},
|
||||
"node_modules/pify": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/pify/-/pify-4.0.1.tgz",
|
||||
"integrity": "sha512-uB80kBFb/tfd68bVleG9T5GGsGPjJrLAUpR5PZIrhBnIaRTQRjqdJSsIKkOP6OAIFbj7GOrcudc5pNjZ+geV2g==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/possible-typed-array-names": {
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/possible-typed-array-names/-/possible-typed-array-names-1.1.0.tgz",
|
||||
"integrity": "sha512-/+5VFTchJDoVj3bhoqi6UeymcD00DAwb1nJwamzPvHEszJ4FpF6SNNbUbOS8yI56qHzdV8eK0qEfOSiodkTdxg==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/postcss": {
|
||||
"version": "8.5.23",
|
||||
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.23.tgz",
|
||||
@@ -5466,15 +5162,6 @@
|
||||
"node": ">= 0.8.0"
|
||||
}
|
||||
},
|
||||
"node_modules/process": {
|
||||
"version": "0.11.10",
|
||||
"resolved": "https://registry.npmjs.org/process/-/process-0.11.10.tgz",
|
||||
"integrity": "sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 0.6.0"
|
||||
}
|
||||
},
|
||||
"node_modules/protobufjs": {
|
||||
"version": "7.6.5",
|
||||
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.5.tgz",
|
||||
@@ -5770,49 +5457,12 @@
|
||||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/set-function-length": {
|
||||
"version": "1.2.2",
|
||||
"resolved": "https://registry.npmjs.org/set-function-length/-/set-function-length-1.2.2.tgz",
|
||||
"integrity": "sha512-pgRc4hJ4/sNjWCSS9AmnS40x3bNMDTknHgL5UaMBTMyJnU90EgWh1Rz+MC9eFu4BuN/UwZjKQuY/1v3rM7HMfg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"define-data-property": "^1.1.4",
|
||||
"es-errors": "^1.3.0",
|
||||
"function-bind": "^1.1.2",
|
||||
"get-intrinsic": "^1.2.4",
|
||||
"gopd": "^1.0.1",
|
||||
"has-property-descriptors": "^1.0.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/setprototypeof": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz",
|
||||
"integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/sha.js": {
|
||||
"version": "2.4.12",
|
||||
"resolved": "https://registry.npmjs.org/sha.js/-/sha.js-2.4.12.tgz",
|
||||
"integrity": "sha512-8LzC5+bvI45BjpfXU8V5fdU2mfeKiQe1D1gIMn7XUlF3OTUrpdJpPPH4EMAnF0DsHHdSZqCdSss5qCmJKuiO3w==",
|
||||
"license": "(MIT AND BSD-3-Clause)",
|
||||
"dependencies": {
|
||||
"inherits": "^2.0.4",
|
||||
"safe-buffer": "^5.2.1",
|
||||
"to-buffer": "^1.2.0"
|
||||
},
|
||||
"bin": {
|
||||
"sha.js": "bin.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/shebang-command": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz",
|
||||
@@ -5915,51 +5565,6 @@
|
||||
"dev": true,
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/simple-concat": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",
|
||||
"integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/simple-get": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",
|
||||
"integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"decompress-response": "^6.0.0",
|
||||
"once": "^1.3.1",
|
||||
"simple-concat": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/simple-update-notifier": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/simple-update-notifier/-/simple-update-notifier-2.0.0.tgz",
|
||||
@@ -6297,20 +5902,6 @@
|
||||
"node": ">=14.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/to-buffer": {
|
||||
"version": "1.2.2",
|
||||
"resolved": "https://registry.npmjs.org/to-buffer/-/to-buffer-1.2.2.tgz",
|
||||
"integrity": "sha512-db0E3UJjcFhpDhAF4tLo03oli3pwl3dbnzXOUIlRKrp+ldk/VUxzpWYZENsw2SZiuBjHAk7DfB0VU7NKdpb6sw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"isarray": "^2.0.5",
|
||||
"safe-buffer": "^5.2.1",
|
||||
"typed-array-buffer": "^1.0.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/to-regex-range": {
|
||||
"version": "5.0.1",
|
||||
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
|
||||
@@ -6456,20 +6047,6 @@
|
||||
"url": "https://opencollective.com/express"
|
||||
}
|
||||
},
|
||||
"node_modules/typed-array-buffer": {
|
||||
"version": "1.0.3",
|
||||
"resolved": "https://registry.npmjs.org/typed-array-buffer/-/typed-array-buffer-1.0.3.tgz",
|
||||
"integrity": "sha512-nAYYwfY3qnzX30IkA6AQZjVbtK6duGontcQm1WSG1MD94YLqK0515GNApXkoxKOWMusVssAHWLh9SeaoefYFGw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"call-bound": "^1.0.3",
|
||||
"es-errors": "^1.3.0",
|
||||
"is-typed-array": "^1.1.14"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/typedarray": {
|
||||
"version": "0.0.6",
|
||||
"resolved": "https://registry.npmjs.org/typedarray/-/typedarray-0.0.6.tgz",
|
||||
@@ -6778,27 +6355,6 @@
|
||||
"node": ">= 8"
|
||||
}
|
||||
},
|
||||
"node_modules/which-typed-array": {
|
||||
"version": "1.1.20",
|
||||
"resolved": "https://registry.npmjs.org/which-typed-array/-/which-typed-array-1.1.20.tgz",
|
||||
"integrity": "sha512-LYfpUkmqwl0h9A2HL09Mms427Q1RZWuOHsukfVcKRq9q95iQxdw0ix1JQrqbcDR9PH1QDwf5Qo8OZb5lksZ8Xg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"available-typed-arrays": "^1.0.7",
|
||||
"call-bind": "^1.0.8",
|
||||
"call-bound": "^1.0.4",
|
||||
"for-each": "^0.3.5",
|
||||
"get-proto": "^1.0.1",
|
||||
"gopd": "^1.2.0",
|
||||
"has-tostringtag": "^1.0.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 0.4"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/why-is-node-running": {
|
||||
"version": "2.3.0",
|
||||
"resolved": "https://registry.npmjs.org/why-is-node-running/-/why-is-node-running-2.3.0.tgz",
|
||||
|
||||
@@ -80,7 +80,6 @@
|
||||
"helmet": "^8.1.0",
|
||||
"http-proxy": "^1.18.1",
|
||||
"http-proxy-middleware": "^4.0.0",
|
||||
"isomorphic-git": "^1.37.5",
|
||||
"jsonwebtoken": "^9.0.3",
|
||||
"ldapts": "^9.0.0",
|
||||
"multer": "^2.1.1",
|
||||
|
||||
@@ -18,6 +18,7 @@ import jwt from 'jsonwebtoken';
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { setupTestDb, cleanupTestDb, TEST_USERNAME, TEST_JWT_SECRET } from './helpers/setupTestDb';
|
||||
import { REF_MAX_LEN } from '../services/git/nativeGitTransport';
|
||||
import { DatabaseService } from '../services/DatabaseService';
|
||||
import { ComposeService } from '../services/ComposeService';
|
||||
import { GitSourceService, GitSourceError } from '../services/GitSourceService';
|
||||
@@ -244,6 +245,21 @@ describe('PUT /api/stacks/:stackName/git-source — max-length caps', () => {
|
||||
expect(res.body.error).toMatch(/branch/i);
|
||||
});
|
||||
|
||||
it('does not reject a branch at the transport limit as too long', async () => {
|
||||
// The route and the transport share one bound, so a branch the route
|
||||
// stores is always one the transport will still fetch. This asserts
|
||||
// the shared side of that: at the limit, length is not the objection.
|
||||
const res = await request(app)
|
||||
.put('/api/stacks/existing-stack/git-source')
|
||||
.set('Authorization', `Bearer ${adminToken()}`)
|
||||
.send({
|
||||
...baseBody,
|
||||
repo_url: 'https://github.com/example/repo.git',
|
||||
branch: 'b'.repeat(REF_MAX_LEN),
|
||||
});
|
||||
expect(String(res.body?.error ?? '')).not.toMatch(/too long/i);
|
||||
});
|
||||
|
||||
it('rejects oversized compose_path', async () => {
|
||||
const res = await request(app)
|
||||
.put('/api/stacks/existing-stack/git-source')
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
* - validateCompose YAML pre-check (empty / non-object / syntax error)
|
||||
* - Token round-trip via upsert: encryption, has_token projection, undefined/null/empty/non-empty semantics
|
||||
* - Apply-matrix rejection (auto_deploy requires auto_apply)
|
||||
* - Error code mapping from isomorphic-git failures (REPO_NOT_FOUND, AUTH_FAILED, BRANCH_NOT_FOUND, NETWORK_TIMEOUT)
|
||||
* - Error code mapping from native-git transport failures (REPO_NOT_FOUND, AUTH_FAILED, BRANCH_NOT_FOUND, NETWORK_TIMEOUT)
|
||||
* - Credential scrubbing in surfaced error messages
|
||||
* - Pending state lifecycle (setPending -> apply clears -> dismissPending clears)
|
||||
* - Webhook debounce enforcement
|
||||
@@ -17,6 +17,7 @@ import fs from 'fs';
|
||||
import path from 'path';
|
||||
import { describe, it, expect, vi, beforeAll, afterAll, beforeEach } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
import type { TransportFailure } from '../services/git/errors';
|
||||
import { GitOpsStore } from '../services/gitops/store';
|
||||
import { GitOpsTransitions } from '../services/gitops/transitions';
|
||||
import {
|
||||
@@ -28,17 +29,22 @@ import {
|
||||
|
||||
// ── Hoisted mocks ──────────────────────────────────────────────────────
|
||||
|
||||
const { mockGitClone, mockGitLog } = vi.hoisted(() => ({
|
||||
const { mockResolveRef, mockFetchAtCommit, mockGitClone, mockGitLog } = vi.hoisted(() => ({
|
||||
mockResolveRef: vi.fn(),
|
||||
mockFetchAtCommit: vi.fn(),
|
||||
mockGitClone: vi.fn(),
|
||||
mockGitLog: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('isomorphic-git', () => {
|
||||
const api = { clone: mockGitClone, log: mockGitLog };
|
||||
return { default: api, clone: mockGitClone, log: mockGitLog };
|
||||
});
|
||||
|
||||
vi.mock('isomorphic-git/http/node', () => ({ default: {} }));
|
||||
// The transport boundary is what gets mocked. mockGitClone/mockGitLog remain
|
||||
// as the fixture layer so every per-test override keeps its meaning: clone
|
||||
// writes files into the checkout dir, log yields the deterministic sha.
|
||||
vi.mock('../services/git/nativeGitTransport', () => ({
|
||||
nativeGitTransport: {
|
||||
resolveRef: mockResolveRef,
|
||||
fetchAtCommit: mockFetchAtCommit,
|
||||
},
|
||||
}));
|
||||
|
||||
|
||||
const {
|
||||
@@ -94,8 +100,11 @@ afterAll(() => {
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
mockResolveRef.mockReset();
|
||||
mockFetchAtCommit.mockReset();
|
||||
mockGitClone.mockReset();
|
||||
mockGitLog.mockReset();
|
||||
wireTransportDefaults();
|
||||
mockCaptureCandidate.mockReset();
|
||||
mockCaptureCandidate.mockImplementation(async () => ({ id: 'rec-test-1' }));
|
||||
mockRecoveryAbandon.mockReset();
|
||||
@@ -121,9 +130,46 @@ beforeEach(() => {
|
||||
// ── Helpers ────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Stub out isomorphic-git so that `clone` writes a minimal compose file into
|
||||
* the caller's temp dir and `log` returns a deterministic commit sha. Returns
|
||||
* the sha so tests can compare.
|
||||
* Default transport wiring: resolveRef defers to the log stub so per-test
|
||||
* overrides of mockGitLog keep controlling the final SHA, and fetchAtCommit
|
||||
* delegates to the clone/log fixture fns, handing clone a `dir` that points
|
||||
* at the workspace checkout.
|
||||
*/
|
||||
function wireTransportDefaults(): void {
|
||||
mockResolveRef.mockImplementation(async () => {
|
||||
const log = await mockGitLog({});
|
||||
const oid = Array.isArray(log) ? log[0]?.oid : undefined;
|
||||
return { commitSha: oid ?? '' };
|
||||
});
|
||||
mockFetchAtCommit.mockImplementation(async (req: { workspaceRoot: string; commitSha: string }) => {
|
||||
const path = await import('path');
|
||||
const { promises: fsp } = await import('fs');
|
||||
const dir = path.join(req.workspaceRoot, 'repo');
|
||||
// The real clone creates the checkout dir; fixture impls may not.
|
||||
await fsp.mkdir(dir, { recursive: true });
|
||||
await mockGitClone({ ...req, dir });
|
||||
const log = await mockGitLog({ dir });
|
||||
if (!Array.isArray(log) || !log.length) {
|
||||
// An empty branch produces no remote ref; mirror the structured
|
||||
// failure the real transport raises for that case.
|
||||
throw { transportFailure: true as const, reason: 'ref-not-found', host: 'unknown', hasToken: false };
|
||||
}
|
||||
return { commitSha: log[0].oid, dir };
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Structured transport failure carrying a real-world git stderr sample, for
|
||||
* exercising the service's classification of native-git failures.
|
||||
*/
|
||||
function gitFailure(stderr: string, hasToken: boolean): TransportFailure {
|
||||
return { transportFailure: true as const, reason: 'exit', stderr, exitCode: 128, host: 'github.com', hasToken };
|
||||
}
|
||||
|
||||
/**
|
||||
* Stub out the clone/log fixtures so that `clone` writes a minimal compose
|
||||
* file into the checkout dir and `log` returns a deterministic commit sha.
|
||||
* Returns the sha so tests can compare.
|
||||
*/
|
||||
function mockSuccessfulClone(options: {
|
||||
compose?: string;
|
||||
@@ -445,7 +491,10 @@ describe('GitSourceService.upsert (encryption + reachability)', () => {
|
||||
});
|
||||
|
||||
it('does not persist when dry-run fetch fails', async () => {
|
||||
mockGitClone.mockRejectedValueOnce(Object.assign(new Error('404 not found'), { code: 'NotFoundError' }));
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: repository 'https://github.com/example/nope.git/' not found",
|
||||
false,
|
||||
));
|
||||
const svc = GitSourceService.getInstance();
|
||||
await expect(svc.upsert({
|
||||
stackName: 'unreachable',
|
||||
@@ -598,123 +647,159 @@ describe('GitSourceService error mapping', () => {
|
||||
composePaths: ['compose.yaml'],
|
||||
};
|
||||
|
||||
it('maps 401 with supplied token to AUTH_FAILED', async () => {
|
||||
// A 401 only means "your token is wrong" when the caller actually sent one.
|
||||
mockGitClone.mockRejectedValueOnce(Object.assign(new Error('HTTP Error: 401 Unauthorized'), {
|
||||
code: 'HttpError',
|
||||
data: { statusCode: 401 },
|
||||
}));
|
||||
it('maps an authentication refusal with supplied token to AUTH_FAILED', async () => {
|
||||
// Auth failure only means "your token is wrong" when the caller actually sent one.
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: Authentication failed for 'https://github.com/example/repo.git/'",
|
||||
true,
|
||||
));
|
||||
await expect(svc().fetchFromGit({ ...fetchParams, token: 'ghp_some_token_value' }))
|
||||
.rejects.toMatchObject({ code: 'AUTH_FAILED' });
|
||||
});
|
||||
|
||||
it('maps 401 without a token to REPO_NOT_FOUND with a private-repo hint', async () => {
|
||||
// GitHub returns 404 for genuinely missing public repos but 401/403 can
|
||||
// also reach us for private repos that the caller did not authenticate
|
||||
// to. Without a supplied token, "check your token" is misleading, so we
|
||||
// surface it as "not found or private" and suggest adding a PAT.
|
||||
mockGitClone.mockRejectedValueOnce(Object.assign(new Error('HTTP Error: 401 Unauthorized'), {
|
||||
code: 'HttpError',
|
||||
data: { statusCode: 401 },
|
||||
}));
|
||||
it('maps a credential prompt without a token to REPO_NOT_FOUND with a private-repo hint', async () => {
|
||||
// Private repos demand credentials; without a supplied token,
|
||||
// "check your token" is misleading, so we surface it as "not found or
|
||||
// private" and suggest adding a PAT (GitHub masks private repos too).
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: could not read Username for 'https://github.com/example/repo.git': terminal prompts disabled",
|
||||
false,
|
||||
));
|
||||
await expect(svc().fetchFromGit(fetchParams))
|
||||
.rejects.toMatchObject({ code: 'REPO_NOT_FOUND', message: expect.stringMatching(/private/i) });
|
||||
});
|
||||
|
||||
it('maps 404 HttpError to REPO_NOT_FOUND (not AUTH_FAILED)', async () => {
|
||||
// Regression: isomorphic-git throws HttpError for every non-2xx, so a
|
||||
// 404 on info/refs was previously misclassified as auth failure.
|
||||
mockGitClone.mockRejectedValueOnce(Object.assign(new Error('HTTP Error: 404 Not Found'), {
|
||||
code: 'HttpError',
|
||||
data: { statusCode: 404 },
|
||||
}));
|
||||
it('maps repository-not-found to REPO_NOT_FOUND (not AUTH_FAILED)', async () => {
|
||||
// Regression guard: a missing repo must never read as an auth problem.
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: repository 'https://github.com/example/repo.git/' not found",
|
||||
false,
|
||||
));
|
||||
await expect(svc().fetchFromGit(fetchParams))
|
||||
.rejects.toMatchObject({ code: 'REPO_NOT_FOUND', message: expect.stringMatching(/private/i) });
|
||||
});
|
||||
|
||||
it('maps 404 with a supplied token to REPO_NOT_FOUND with a token-scope hint', async () => {
|
||||
// GitHub returns 404 for both "missing repo" and "token lacks access",
|
||||
// so when the caller did supply a token we point them at URL + scopes
|
||||
// instead of "add a PAT" (which they already did).
|
||||
mockGitClone.mockRejectedValueOnce(Object.assign(new Error('HTTP Error: 404 Not Found'), {
|
||||
code: 'HttpError',
|
||||
data: { statusCode: 404 },
|
||||
}));
|
||||
it('maps repository-not-found with a supplied token to REPO_NOT_FOUND with a token-scope hint', async () => {
|
||||
// GitHub returns not-found for both "missing repo" and "token lacks
|
||||
// access", so when the caller did supply a token we point them at URL
|
||||
// + scopes instead of "add a PAT" (which they already did).
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: repository 'https://github.com/example/repo.git/' not found",
|
||||
true,
|
||||
));
|
||||
await expect(svc().fetchFromGit({ ...fetchParams, token: 'ghp_some_token_value' }))
|
||||
.rejects.toMatchObject({ code: 'REPO_NOT_FOUND', message: expect.stringMatching(/token has read access/i) });
|
||||
});
|
||||
|
||||
it('maps 404/not-found errors to REPO_NOT_FOUND', async () => {
|
||||
mockGitClone.mockRejectedValueOnce(Object.assign(new Error('Repository not found'), { code: 'NotFoundError' }));
|
||||
it('classifies resolve-phase failures too (ls-remote runs before clone)', async () => {
|
||||
// The first real-world failure point is resolution; if the service
|
||||
// ever stops translating its failures this goes generic GIT_ERROR.
|
||||
mockResolveRef.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: Authentication failed for 'https://github.com/example/repo.git/'",
|
||||
true,
|
||||
));
|
||||
await expect(svc().fetchFromGit({ ...fetchParams, token: 'ghp_some_token_value' }))
|
||||
.rejects.toMatchObject({ code: 'AUTH_FAILED' });
|
||||
});
|
||||
|
||||
it('threads the resolved commit, ref, and token into the pinned fetch', async () => {
|
||||
const sha = mockSuccessfulClone();
|
||||
await svc().fetchFromGit({ ...fetchParams, token: 'tok-abc' });
|
||||
expect(mockFetchAtCommit.mock.calls[0][0]).toMatchObject({
|
||||
commitSha: sha,
|
||||
ref: 'main',
|
||||
repoUrl: 'https://github.com/example/repo.git',
|
||||
token: 'tok-abc',
|
||||
workspaceRoot: expect.any(String),
|
||||
});
|
||||
});
|
||||
|
||||
it('removes the transport workspace after success and after failure', async () => {
|
||||
const fsMod = await import('fs');
|
||||
mockSuccessfulClone();
|
||||
await svc().fetchFromGit(fetchParams);
|
||||
const successRoot = mockFetchAtCommit.mock.calls[0][0].workspaceRoot;
|
||||
expect(fsMod.existsSync(successRoot)).toBe(false);
|
||||
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure('fatal: repository not found', false));
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'REPO_NOT_FOUND' });
|
||||
const failureRoot = mockFetchAtCommit.mock.calls[1][0].workspaceRoot;
|
||||
expect(fsMod.existsSync(failureRoot)).toBe(false);
|
||||
});
|
||||
|
||||
it('reports BRANCH_NOT_FOUND for a branch with no commits', async () => {
|
||||
// Resolve-first turns an empty branch into a missing remote head.
|
||||
mockGitLog.mockResolvedValue([]);
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({
|
||||
code: 'BRANCH_NOT_FOUND',
|
||||
message: expect.stringMatching(/Branch not found/),
|
||||
});
|
||||
});
|
||||
|
||||
it('maps short not-found phrasing to REPO_NOT_FOUND', async () => {
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure('fatal: repository not found', false));
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'REPO_NOT_FOUND' });
|
||||
});
|
||||
|
||||
it('maps resolve-ref errors to BRANCH_NOT_FOUND', async () => {
|
||||
// Message phrased to miss the REPO_NOT_FOUND regex ("could not resolve")
|
||||
// so the BRANCH_NOT_FOUND branch is exercised.
|
||||
mockGitClone.mockRejectedValueOnce(Object.assign(new Error('unknown ref nonexistent'), { code: 'ResolveRefError' }));
|
||||
it('maps remote-branch-not-found to BRANCH_NOT_FOUND', async () => {
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
'fatal: Remote branch nonexistent not found in upstream origin',
|
||||
false,
|
||||
));
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'BRANCH_NOT_FOUND' });
|
||||
});
|
||||
|
||||
it('maps timeout errors to NETWORK_TIMEOUT', async () => {
|
||||
mockGitClone.mockRejectedValueOnce(new Error('ETIMEDOUT connecting to host'));
|
||||
it('maps connection timeouts to NETWORK_TIMEOUT', async () => {
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: unable to access 'https://github.com/example/repo.git/': Failed to connect to github.com port 443 after 21005 ms: Connection timed out",
|
||||
false,
|
||||
));
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'NETWORK_TIMEOUT' });
|
||||
});
|
||||
|
||||
it('maps a bare "fetch failed" TypeError with an ENOTFOUND cause to NETWORK_TIMEOUT', async () => {
|
||||
// Node's global fetch() reports DNS failure as TypeError('fetch failed')
|
||||
// with the real reason on err.cause. Without cause-unwrapping this fell
|
||||
// through to a useless GIT_ERROR: "fetch failed".
|
||||
mockGitClone.mockRejectedValueOnce(
|
||||
new TypeError('fetch failed', {
|
||||
cause: Object.assign(new Error('getaddrinfo ENOTFOUND github.com'), { code: 'ENOTFOUND' }),
|
||||
}),
|
||||
);
|
||||
it('maps DNS failure stderr to NETWORK_TIMEOUT', async () => {
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: unable to access 'https://github.com/example/repo.git/': Could not resolve host: github.com",
|
||||
false,
|
||||
));
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'NETWORK_TIMEOUT' });
|
||||
});
|
||||
|
||||
it('maps a "fetch failed" TypeError with an ECONNREFUSED cause to NETWORK_TIMEOUT', async () => {
|
||||
mockGitClone.mockRejectedValueOnce(
|
||||
new TypeError('fetch failed', {
|
||||
cause: Object.assign(new Error('connect ECONNREFUSED 127.0.0.1:443'), { code: 'ECONNREFUSED' }),
|
||||
}),
|
||||
);
|
||||
it('maps connection-refused stderr to NETWORK_TIMEOUT', async () => {
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: unable to access 'https://github.com/example/repo.git/': Failed to connect to github.com port 443: Connection refused",
|
||||
false,
|
||||
));
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'NETWORK_TIMEOUT' });
|
||||
});
|
||||
|
||||
it('surfaces the host instead of bare "fetch failed" in transport errors', async () => {
|
||||
mockGitClone.mockRejectedValueOnce(
|
||||
new TypeError('fetch failed', {
|
||||
cause: Object.assign(new Error('getaddrinfo ENOTFOUND'), { code: 'ENOTFOUND' }),
|
||||
}),
|
||||
);
|
||||
it('surfaces the host in DNS transport errors', async () => {
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: unable to access 'https://github.com/example/repo.git/': Could not resolve host: github.com",
|
||||
false,
|
||||
));
|
||||
try {
|
||||
await svc().fetchFromGit(fetchParams);
|
||||
expect.fail('should have thrown');
|
||||
} catch (e) {
|
||||
const err = e as Error;
|
||||
expect(err.message).not.toMatch(/^fetch failed$/i);
|
||||
expect(err.message).toContain('github.com');
|
||||
}
|
||||
});
|
||||
|
||||
it('unwraps a nested fetch cause chain to find the transport code', async () => {
|
||||
mockGitClone.mockRejectedValueOnce(
|
||||
new TypeError('fetch failed', {
|
||||
cause: new TypeError('terminated', {
|
||||
cause: Object.assign(new Error('reset'), { code: 'ECONNRESET' }),
|
||||
}),
|
||||
}),
|
||||
);
|
||||
it('maps a reset connection to NETWORK_TIMEOUT', async () => {
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
'fatal: the remote end hung up unexpectedly',
|
||||
false,
|
||||
));
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({ code: 'NETWORK_TIMEOUT' });
|
||||
});
|
||||
|
||||
it('maps a TLS certificate "fetch failed" cause to a certificate GIT_ERROR', async () => {
|
||||
mockGitClone.mockRejectedValueOnce(
|
||||
new TypeError('fetch failed', {
|
||||
cause: Object.assign(new Error('self-signed certificate'), { code: 'DEPTH_ZERO_SELF_SIGNED_CERT' }),
|
||||
}),
|
||||
);
|
||||
it('maps a TLS certificate failure to a certificate GIT_ERROR', async () => {
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: unable to access 'https://github.com/example/repo.git/': SSL certificate problem: self-signed certificate",
|
||||
false,
|
||||
));
|
||||
await expect(svc().fetchFromGit(fetchParams)).rejects.toMatchObject({
|
||||
code: 'GIT_ERROR',
|
||||
message: expect.stringMatching(/certificate/i),
|
||||
@@ -731,7 +816,10 @@ describe('GitSourceService error mapping', () => {
|
||||
});
|
||||
|
||||
it('scrubs inline credentials from surfaced error messages', async () => {
|
||||
mockGitClone.mockRejectedValueOnce(new Error('Failed: https://user:supersecret@github.com/example/repo.git 500'));
|
||||
mockFetchAtCommit.mockRejectedValueOnce(gitFailure(
|
||||
"fatal: unable to access 'https://user:supersecret@github.com/example/repo.git/': The requested URL returned error: 500",
|
||||
false,
|
||||
));
|
||||
try {
|
||||
await svc().fetchFromGit(fetchParams);
|
||||
expect.fail('should have thrown');
|
||||
@@ -743,42 +831,6 @@ describe('GitSourceService error mapping', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('countingBodyIterator (clone size cap)', () => {
|
||||
function chunkStream(...sizes: number[]): AsyncIterableIterator<Uint8Array> {
|
||||
async function* gen(): AsyncIterableIterator<Uint8Array> {
|
||||
for (const s of sizes) yield new Uint8Array(s);
|
||||
}
|
||||
return gen();
|
||||
}
|
||||
|
||||
it('passes chunks through unchanged while under the cap', async () => {
|
||||
const { countingBodyIterator } = await import('../services/GitSourceService');
|
||||
const controller = new AbortController();
|
||||
const state = { exceeded: false, received: 0 };
|
||||
const out: number[] = [];
|
||||
for await (const c of countingBodyIterator(chunkStream(10, 20, 30), controller, 1000, state)) {
|
||||
out.push(c.byteLength);
|
||||
}
|
||||
expect(out).toEqual([10, 20, 30]);
|
||||
expect(state.exceeded).toBe(false);
|
||||
expect(state.received).toBe(60);
|
||||
expect(controller.signal.aborted).toBe(false);
|
||||
});
|
||||
|
||||
it('aborts the transport and throws once the cumulative size exceeds the cap', async () => {
|
||||
const { countingBodyIterator } = await import('../services/GitSourceService');
|
||||
const controller = new AbortController();
|
||||
const state = { exceeded: false, received: 0 };
|
||||
await expect((async () => {
|
||||
for await (const _c of countingBodyIterator(chunkStream(60, 60), controller, 100, state)) {
|
||||
void _c;
|
||||
}
|
||||
})()).rejects.toThrow(/maximum allowed size/i);
|
||||
expect(state.exceeded).toBe(true);
|
||||
expect(controller.signal.aborted).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GitSourceService.fetchFromGit (size limits)', () => {
|
||||
const svc = () => GitSourceService.getInstance();
|
||||
const fetchParams = {
|
||||
@@ -788,8 +840,8 @@ describe('GitSourceService.fetchFromGit (size limits)', () => {
|
||||
};
|
||||
|
||||
it('rejects a compose file larger than the per-file read cap', async () => {
|
||||
// The download cap bounds the compressed pack, not a single decompressed
|
||||
// file, so readRepoFile guards the in-memory read by file size.
|
||||
// The workspace cap bounds the on-disk clone, not a single file, so
|
||||
// readRepoFile guards the in-memory read by file size.
|
||||
mockSuccessfulClone();
|
||||
const { promises: fsp } = await import('fs');
|
||||
const lstatSpy = vi.spyOn(fsp, 'lstat').mockResolvedValue({
|
||||
@@ -805,19 +857,14 @@ describe('GitSourceService.fetchFromGit (size limits)', () => {
|
||||
lstatSpy.mockRestore();
|
||||
});
|
||||
|
||||
it('surfaces a clone-size error when the download exceeds the cap', async () => {
|
||||
// Drive the real size-counting transport the service injected into
|
||||
// git.clone, with a tiny cap, and confirm fetchFromGit reports it as a
|
||||
// clone-size error rather than a generic transport failure.
|
||||
it('surfaces a clone-size error and forwards the configured cap to the transport', async () => {
|
||||
// The transport enforces the cap with its size watchdog (covered in the
|
||||
// transport unit tests); here we pin the plumbing: the env knob reaches
|
||||
// the transport as maxBytes, and a structured size failure translates
|
||||
// into the clone-size message rather than a generic transport error.
|
||||
process.env.GITSOURCE_MAX_CLONE_BYTES = '8';
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue(
|
||||
new Response(new Uint8Array(64), { status: 200 }),
|
||||
);
|
||||
mockGitClone.mockImplementation(async (args: {
|
||||
http: { request: (r: { url: string; method: string; headers: Record<string, string> }) => Promise<{ body: AsyncIterableIterator<Uint8Array> }> };
|
||||
}) => {
|
||||
const resp = await args.http.request({ url: 'https://example.test/info/refs', method: 'GET', headers: {} });
|
||||
for await (const chunk of resp.body) { void chunk; }
|
||||
mockFetchAtCommit.mockImplementationOnce(async () => {
|
||||
throw { transportFailure: true as const, reason: 'size', maxBytes: 8, host: 'github.com', hasToken: false };
|
||||
});
|
||||
|
||||
try {
|
||||
@@ -825,9 +872,9 @@ describe('GitSourceService.fetchFromGit (size limits)', () => {
|
||||
code: 'GIT_ERROR',
|
||||
message: expect.stringMatching(/exceeds the maximum clone size/i),
|
||||
});
|
||||
expect(mockFetchAtCommit.mock.calls[0][0]).toMatchObject({ maxBytes: 8 });
|
||||
} finally {
|
||||
delete process.env.GITSOURCE_MAX_CLONE_BYTES;
|
||||
fetchSpy.mockRestore();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,280 @@
|
||||
/**
|
||||
* Real end-to-end coverage for authenticated native-git transport.
|
||||
*
|
||||
* Every other transport test mocks `child_process` or bypasses the
|
||||
* transport module entirely, so nothing proves the credential helper, the
|
||||
* `x-access-token` username convention, argv quoting, and env-var handoff
|
||||
* actually work against a real git binary talking to a server that checks
|
||||
* Basic Auth. This file does: a local HTTPS smart-HTTP server that requires
|
||||
* a token and rejects everything else, driven through the real
|
||||
* `nativeGitTransport` with nothing mocked.
|
||||
*
|
||||
* Reuses the committed dev-only TLS fixture from the Git Sources E2E specs
|
||||
* (e2e/fixtures/git-ca.pem / git-server.pem|key) via direct file reads
|
||||
* rather than importing e2e/gitServer.helper.ts: backend's tsconfig pins
|
||||
* rootDir to backend/src, so a cross-directory import would fail `tsc
|
||||
* --noEmit`.
|
||||
*
|
||||
* Soft-skips when the system git binary is unavailable, mirroring the E2E
|
||||
* fixture server's own skip.
|
||||
*/
|
||||
import { spawn, spawnSync } from 'child_process';
|
||||
import { promises as fs, mkdtempSync, readFileSync, writeFileSync } from 'fs';
|
||||
import https from 'https';
|
||||
import os from 'os';
|
||||
import path from 'path';
|
||||
import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest';
|
||||
import { classifyGitFailure, isTransportFailure } from '../services/git/errors';
|
||||
import { nativeGitTransport } from '../services/git/nativeGitTransport';
|
||||
|
||||
function gitAvailable(): boolean {
|
||||
return spawnSync('git', ['--version'], { stdio: 'ignore' }).status === 0;
|
||||
}
|
||||
|
||||
const FIXTURES_DIR = path.resolve(__dirname, '..', '..', '..', 'e2e', 'fixtures');
|
||||
const VALID_TOKEN = 'sencho-integration-test-token-do-not-leak';
|
||||
const FILE_CONTENT = 'hello from the authenticated fixture repo\n';
|
||||
|
||||
/**
|
||||
* Build a bare repo with one committed file. Mirrors e2e/gitServer.helper.ts's
|
||||
* fixture builder. Returns both the served bare dir and every scratch
|
||||
* directory created along the way, so the caller can remove them all.
|
||||
*/
|
||||
function buildBareFixtureRepo(): { bareDir: string; scratchDirs: string[] } {
|
||||
const srcDir = mkdtempSync(path.join(os.tmpdir(), 'sencho-git-auth-src-'));
|
||||
writeFileSync(path.join(srcDir, 'hello.txt'), FILE_CONTENT);
|
||||
const run = (args: string[]) => {
|
||||
const r = spawnSync('git', args, { cwd: srcDir, encoding: 'utf8' });
|
||||
if (r.status !== 0) throw new Error(`git ${args[0]} failed: ${r.stderr}`);
|
||||
};
|
||||
run(['init', '-b', 'main']);
|
||||
run(['config', 'user.email', 'integration-test@sencho.test']);
|
||||
run(['config', 'user.name', 'Sencho Integration Test']);
|
||||
run(['add', '-A']);
|
||||
// Explicitly off: a developer machine or CI runner with commit.gpgsign=true
|
||||
// in its global gitconfig would otherwise fail this fixture commit.
|
||||
run(['-c', 'commit.gpgsign=false', 'commit', '-m', 'fixture']);
|
||||
|
||||
const bareRoot = mkdtempSync(path.join(os.tmpdir(), 'sencho-git-auth-bare-'));
|
||||
const bareDir = path.join(bareRoot, 'repo.git');
|
||||
const clone = spawnSync('git', ['clone', '--bare', '--quiet', srcDir, bareDir], { encoding: 'utf8' });
|
||||
if (clone.status !== 0) throw new Error(`git clone --bare failed: ${clone.stderr}`);
|
||||
return { bareDir, scratchDirs: [srcDir, bareRoot] };
|
||||
}
|
||||
|
||||
/** Serve one bare repo over HTTPS smart-HTTP, rejecting any request without a valid Basic Auth token. */
|
||||
function serveAuthedRepo(bareDir: string): Promise<{ url: string; close: () => void }> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const expectedAuth = `Basic ${Buffer.from(`x-access-token:${VALID_TOKEN}`).toString('base64')}`;
|
||||
const server = https.createServer(
|
||||
{
|
||||
cert: readFileSync(path.join(FIXTURES_DIR, 'git-server.pem')),
|
||||
key: readFileSync(path.join(FIXTURES_DIR, 'git-server.key')),
|
||||
},
|
||||
(req, res) => {
|
||||
if (req.headers.authorization !== expectedAuth) {
|
||||
res.statusCode = 401;
|
||||
res.setHeader('WWW-Authenticate', 'Basic realm="sencho-integration-test"');
|
||||
res.end('authentication required');
|
||||
return;
|
||||
}
|
||||
const url = req.url ?? '/';
|
||||
if (!url.startsWith('/repo.git/')) {
|
||||
res.statusCode = 404;
|
||||
res.end('unknown repo');
|
||||
return;
|
||||
}
|
||||
const pathname = url.slice('/repo.git'.length).split('?')[0];
|
||||
if (pathname === '/info/refs' && (req.method === 'GET' || req.method === 'POST')) {
|
||||
const ps = spawn('git', ['upload-pack', '--stateless-rpc', '--advertise-refs', bareDir]);
|
||||
let out = Buffer.alloc(0);
|
||||
ps.stdout.on('data', (d: Buffer) => {
|
||||
out = Buffer.concat([out, d]);
|
||||
});
|
||||
ps.on('close', (code) => {
|
||||
if (code !== 0) {
|
||||
res.statusCode = 500;
|
||||
res.end('git upload-pack failed');
|
||||
return;
|
||||
}
|
||||
res.setHeader('content-type', 'application/x-git-upload-pack-advertisement');
|
||||
res.end(Buffer.concat([Buffer.from('001e# service=git-upload-pack\n0000'), out]));
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (pathname === '/git-upload-pack' && req.method === 'POST') {
|
||||
const ps = spawn('git', ['upload-pack', '--stateless-rpc', bareDir]);
|
||||
res.setHeader('content-type', 'application/x-git-upload-pack-result');
|
||||
ps.stdout.pipe(res);
|
||||
ps.stdin.on('error', (err) => {
|
||||
// EPIPE/ECONNRESET: the client aborted mid-stream.
|
||||
// Anything else is a real bug in this fixture server.
|
||||
const code = (err as NodeJS.ErrnoException).code;
|
||||
if (code !== 'EPIPE' && code !== 'ECONNRESET') throw err;
|
||||
});
|
||||
req.pipe(ps.stdin);
|
||||
return;
|
||||
}
|
||||
res.statusCode = 404;
|
||||
res.end('unsupported git endpoint');
|
||||
},
|
||||
);
|
||||
server.on('error', reject);
|
||||
server.listen(0, '127.0.0.1', () => {
|
||||
const address = server.address();
|
||||
if (address === null || typeof address === 'string') {
|
||||
reject(new Error('server did not bind'));
|
||||
return;
|
||||
}
|
||||
resolve({ url: `https://127.0.0.1:${address.port}/repo.git`, close: () => server.close() });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
describe.skipIf(!gitAvailable())('authenticated native git transport (real git, real TLS, real auth)', () => {
|
||||
let repoUrl: string;
|
||||
let closeServer: () => void;
|
||||
let prevExtraCaCerts: string | undefined;
|
||||
let fixtureScratchDirs: string[] = [];
|
||||
const workspaces: string[] = [];
|
||||
|
||||
beforeAll(async () => {
|
||||
const { bareDir, scratchDirs } = buildBareFixtureRepo();
|
||||
fixtureScratchDirs = scratchDirs;
|
||||
const served = await serveAuthedRepo(bareDir);
|
||||
repoUrl = served.url;
|
||||
closeServer = served.close;
|
||||
prevExtraCaCerts = process.env.NODE_EXTRA_CA_CERTS;
|
||||
process.env.NODE_EXTRA_CA_CERTS = path.join(FIXTURES_DIR, 'git-ca.pem');
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
closeServer?.();
|
||||
if (prevExtraCaCerts === undefined) delete process.env.NODE_EXTRA_CA_CERTS;
|
||||
else process.env.NODE_EXTRA_CA_CERTS = prevExtraCaCerts;
|
||||
await Promise.all(fixtureScratchDirs.map((d) => fs.rm(d, { recursive: true, force: true })));
|
||||
});
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(workspaces.splice(0).map((w) => fs.rm(w, { recursive: true, force: true })));
|
||||
});
|
||||
|
||||
async function makeWorkspace(): Promise<string> {
|
||||
const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'sencho-git-auth-ws-'));
|
||||
workspaces.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
/**
|
||||
* A workspace nested under a directory whose name contains a space, plus
|
||||
* the other characters git's shell treats specially. Git reads
|
||||
* `credential.helper` as a shell string, so a transport that interpolates
|
||||
* the helper's path into it breaks here (and on any host whose temp dir
|
||||
* sits under something like `C:/Users/Ada Lovelace/...`) while passing
|
||||
* every normal-path test.
|
||||
*/
|
||||
async function makeAwkwardWorkspace(): Promise<string> {
|
||||
const parent = await fs.mkdtemp(path.join(os.tmpdir(), 'sencho-git-auth-odd-'));
|
||||
workspaces.push(parent);
|
||||
const dir = path.join(parent, "a dir with spaces & 'quotes' $dollar");
|
||||
await fs.mkdir(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
it('clones a private repo end-to-end with a valid token', async () => {
|
||||
const workspaceRoot = await makeWorkspace();
|
||||
const resolved = await nativeGitTransport.resolveRef({
|
||||
repoUrl,
|
||||
ref: 'main',
|
||||
token: VALID_TOKEN,
|
||||
timeoutMs: 15_000,
|
||||
workspaceRoot,
|
||||
});
|
||||
expect(resolved.commitSha).toMatch(/^[0-9a-f]{40}$/);
|
||||
|
||||
const fetchWorkspace = await makeWorkspace();
|
||||
const fetched = await nativeGitTransport.fetchAtCommit({
|
||||
repoUrl,
|
||||
ref: 'main',
|
||||
token: VALID_TOKEN,
|
||||
commitSha: resolved.commitSha,
|
||||
timeoutMs: 15_000,
|
||||
workspaceRoot: fetchWorkspace,
|
||||
maxBytes: 10 * 1024 * 1024,
|
||||
});
|
||||
expect(fetched.commitSha).toBe(resolved.commitSha);
|
||||
const content = await fs.readFile(path.join(fetched.dir, 'hello.txt'), 'utf8');
|
||||
expect(content).toBe(FILE_CONTENT);
|
||||
});
|
||||
|
||||
it('clones a private repo end-to-end from a workspace path containing spaces and shell metacharacters', async () => {
|
||||
const workspaceRoot = await makeAwkwardWorkspace();
|
||||
const resolved = await nativeGitTransport.resolveRef({
|
||||
repoUrl,
|
||||
ref: 'main',
|
||||
token: VALID_TOKEN,
|
||||
timeoutMs: 15_000,
|
||||
workspaceRoot,
|
||||
});
|
||||
expect(resolved.commitSha).toMatch(/^[0-9a-f]{40}$/);
|
||||
|
||||
const fetchWorkspace = await makeAwkwardWorkspace();
|
||||
const fetched = await nativeGitTransport.fetchAtCommit({
|
||||
repoUrl,
|
||||
ref: 'main',
|
||||
token: VALID_TOKEN,
|
||||
commitSha: resolved.commitSha,
|
||||
timeoutMs: 15_000,
|
||||
workspaceRoot: fetchWorkspace,
|
||||
maxBytes: 10 * 1024 * 1024,
|
||||
});
|
||||
expect(fetched.commitSha).toBe(resolved.commitSha);
|
||||
expect(await fs.readFile(path.join(fetched.dir, 'hello.txt'), 'utf8')).toBe(FILE_CONTENT);
|
||||
});
|
||||
|
||||
it('still classifies a wrong token as AUTH_FAILED from an awkward workspace path', async () => {
|
||||
// Guards the subtler half of the same defect: when the helper cannot
|
||||
// execute, git sends no credentials at all and the server's 401 reads
|
||||
// like an anonymous request, so the failure silently downgrades to the
|
||||
// private-repo masking classification instead of AUTH_FAILED.
|
||||
const workspaceRoot = await makeAwkwardWorkspace();
|
||||
const failure = await nativeGitTransport
|
||||
.resolveRef({ repoUrl, ref: 'main', token: 'wrong-token', timeoutMs: 15_000, workspaceRoot })
|
||||
.then(() => null, (e: unknown) => e);
|
||||
|
||||
expect(isTransportFailure(failure)).toBe(true);
|
||||
if (!isTransportFailure(failure)) throw new Error('unreachable');
|
||||
expect(failure.hasToken).toBe(true);
|
||||
expect(classifyGitFailure(failure).code).toBe('AUTH_FAILED');
|
||||
});
|
||||
|
||||
it('fails with the private-repo masking classification when no token is supplied', async () => {
|
||||
const workspaceRoot = await makeWorkspace();
|
||||
const failure = await nativeGitTransport
|
||||
.resolveRef({ repoUrl, ref: 'main', timeoutMs: 15_000, workspaceRoot })
|
||||
.then(() => null, (e: unknown) => e);
|
||||
|
||||
expect(isTransportFailure(failure)).toBe(true);
|
||||
if (!isTransportFailure(failure)) throw new Error('unreachable');
|
||||
expect(failure.hasToken).toBe(false);
|
||||
expect(classifyGitFailure(failure).code).toBe('REPO_NOT_FOUND');
|
||||
});
|
||||
|
||||
it('fails with AUTH_FAILED when an invalid token is supplied, and never leaks it', async () => {
|
||||
const wrongToken = 'this-token-is-wrong-and-must-never-appear-in-output';
|
||||
const workspaceRoot = await makeWorkspace();
|
||||
const failure = await nativeGitTransport
|
||||
.resolveRef({ repoUrl, ref: 'main', token: wrongToken, timeoutMs: 15_000, workspaceRoot })
|
||||
.then(() => null, (e: unknown) => e);
|
||||
|
||||
expect(isTransportFailure(failure)).toBe(true);
|
||||
if (!isTransportFailure(failure)) throw new Error('unreachable');
|
||||
expect(failure.hasToken).toBe(true);
|
||||
const classified = classifyGitFailure(failure);
|
||||
expect(classified.code).toBe('AUTH_FAILED');
|
||||
|
||||
const serialized = JSON.stringify(failure) + classified.message;
|
||||
expect(serialized).not.toContain(wrongToken);
|
||||
expect(serialized).not.toContain(VALID_TOKEN);
|
||||
});
|
||||
});
|
||||
File diff suppressed because it is too large
Load Diff
@@ -21,18 +21,25 @@ import path from 'path';
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { setupTestDb, cleanupTestDb } from './helpers/setupTestDb';
|
||||
|
||||
const { mockGitClone, mockGitLog, compose } = vi.hoisted(() => ({
|
||||
const { mockResolveRef, mockFetchAtCommit, mockGitClone, mockGitLog, compose } = vi.hoisted(() => ({
|
||||
mockResolveRef: vi.fn(),
|
||||
mockFetchAtCommit: vi.fn(),
|
||||
mockGitClone: vi.fn(),
|
||||
mockGitLog: vi.fn(),
|
||||
/** Exit code the next daemon-dependent compose command reports. */
|
||||
compose: { exitCode: 1 },
|
||||
}));
|
||||
|
||||
vi.mock('isomorphic-git', () => {
|
||||
const api = { clone: mockGitClone, log: mockGitLog };
|
||||
return { default: api, clone: mockGitClone, log: mockGitLog };
|
||||
});
|
||||
vi.mock('isomorphic-git/http/node', () => ({ default: {} }));
|
||||
// The transport boundary is what gets mocked (see git-source-service.test.ts,
|
||||
// which established this seam). mockGitClone/mockGitLog remain as the
|
||||
// fixture layer so stageRepo() keeps its meaning: clone writes files into
|
||||
// the checkout dir, log yields the deterministic sha.
|
||||
vi.mock('../services/git/nativeGitTransport', () => ({
|
||||
nativeGitTransport: {
|
||||
resolveRef: mockResolveRef,
|
||||
fetchAtCommit: mockFetchAtCommit,
|
||||
},
|
||||
}));
|
||||
|
||||
/**
|
||||
* Compose verbs that need a running daemon and are issued through `spawn`.
|
||||
@@ -132,6 +139,33 @@ let GitOpsStore: typeof import('../services/gitops/store').GitOpsStore;
|
||||
let GitOpsTransitions: typeof import('../services/gitops/transitions').GitOpsTransitions;
|
||||
let projectApplication: typeof import('../services/gitops/derive').projectApplication;
|
||||
|
||||
/**
|
||||
* Default transport wiring: resolveRef defers to the log stub so per-test
|
||||
* overrides of mockGitLog keep controlling the final SHA, and fetchAtCommit
|
||||
* delegates to the clone/log fixture fns, handing clone a `dir` that points
|
||||
* at the workspace checkout. Mirrors git-source-service.test.ts's
|
||||
* wireTransportDefaults.
|
||||
*/
|
||||
function wireTransportDefaults(): void {
|
||||
mockResolveRef.mockImplementation(async () => {
|
||||
const log = await mockGitLog({});
|
||||
const oid = Array.isArray(log) ? log[0]?.oid : undefined;
|
||||
return { commitSha: oid ?? '' };
|
||||
});
|
||||
mockFetchAtCommit.mockImplementation(async (req: { workspaceRoot: string; commitSha: string }) => {
|
||||
const dir = path.join(req.workspaceRoot, 'repo');
|
||||
await fsPromises.mkdir(dir, { recursive: true });
|
||||
await mockGitClone({ ...req, dir });
|
||||
const log = await mockGitLog({ dir });
|
||||
if (!Array.isArray(log) || !log.length) {
|
||||
// An empty branch produces no remote ref; mirror the structured
|
||||
// failure the real transport raises for that case.
|
||||
throw { transportFailure: true as const, reason: 'ref-not-found', host: 'unknown', hasToken: false };
|
||||
}
|
||||
return { commitSha: log[0].oid, dir };
|
||||
});
|
||||
}
|
||||
|
||||
/** Make the next clone produce a project containing this compose content. */
|
||||
function stageRepo(content: string, sha: string, extraFiles: Record<string, string> = {}): void {
|
||||
mockGitClone.mockImplementation(async ({ dir }: { dir: string }) => {
|
||||
@@ -166,8 +200,11 @@ describe('Direct Git producers drive the revision state', () => {
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
mockResolveRef.mockReset();
|
||||
mockFetchAtCommit.mockReset();
|
||||
mockGitClone.mockReset();
|
||||
mockGitLog.mockReset();
|
||||
wireTransportDefaults();
|
||||
compose.exitCode = 1;
|
||||
});
|
||||
|
||||
|
||||
@@ -16,10 +16,13 @@ import { isValidGitSourcePath, isValidStackName } from '../utils/validation';
|
||||
import { sendGitSourceError, webhookPullStatus } from '../utils/gitSourceHttp';
|
||||
import { sanitizeForLog } from '../utils/safeLog';
|
||||
import { repoUrlRejectionMessage } from '../services/gitops/repoIdentity';
|
||||
import { REF_MAX_LEN } from '../services/git/nativeGitTransport';
|
||||
|
||||
// Reasonable upper bounds so a caller cannot flood the service with huge
|
||||
// payloads. Generous compared to anything a real Git provider emits.
|
||||
const MAX_BRANCH_LENGTH = 256;
|
||||
// The branch bound comes from the transport that ultimately fetches the ref,
|
||||
// so a branch this route accepts can never be refused later as too long.
|
||||
const MAX_BRANCH_LENGTH = REF_MAX_LEN;
|
||||
const MAX_ENV_PATH_LENGTH = 1024;
|
||||
const MAX_TOKEN_LENGTH = 8192;
|
||||
|
||||
|
||||
@@ -24,6 +24,7 @@ import {
|
||||
} from '../services/UpdatePreviewService';
|
||||
import { GitSourceService, GitSourceError, repoHost as gitRepoHost } from '../services/GitSourceService';
|
||||
import { repoUrlRejectionMessage } from '../services/gitops/repoIdentity';
|
||||
import { REF_MAX_LEN } from '../services/git/nativeGitTransport';
|
||||
import { enforcePolicyPreDeploy } from '../services/PolicyEnforcement';
|
||||
import { buildStackDriftReport, type DriftFindingKind, type StackDriftReport } from '../services/DriftDetectionService';
|
||||
import { DriftLedgerService, type DriftTemporal } from '../services/DriftLedgerService';
|
||||
@@ -1117,7 +1118,7 @@ stacksRouter.post('/from-git', async (req: Request, res: Response) => {
|
||||
if (repoUrlError) {
|
||||
return res.status(400).json({ error: repoUrlError });
|
||||
}
|
||||
if (branch.length > 256) {
|
||||
if (branch.length > REF_MAX_LEN) {
|
||||
return res.status(400).json({ error: 'branch is too long' });
|
||||
}
|
||||
if (typeof env_path === 'string' && env_path.length > 1024) {
|
||||
|
||||
@@ -28,6 +28,8 @@ import type { ComposeInputEntry, GitProjectManifest, GitSourceManifestState, Inv
|
||||
import type { GitChangePlan, PublicGitChangePlan, GitChangePlanCounts, PublicGitChangePlanOperation } from '../types/gitChangePlan';
|
||||
import { GIT_CHANGE_PLAN_SCHEMA_VERSION } from '../types/gitChangePlan';
|
||||
import type { NotificationCategory } from './NotificationService';
|
||||
import { classifyGitFailure, isTransportFailure } from './git/errors';
|
||||
import { nativeGitTransport } from './git/nativeGitTransport';
|
||||
import { GitOpsStore } from './gitops/store';
|
||||
import { GitOpsTransitions, GitOpsTransitionError } from './gitops/transitions';
|
||||
import {
|
||||
@@ -42,136 +44,6 @@ import type { GitOpsApplicationRow } from './gitops/types';
|
||||
import { appliedRelPathFor, candidateRelPathForSha, deleteStagingMarker, readStagingMarker, writeStagingMarker } from './gitops/createStagingMarker';
|
||||
import { cleanupUnclaimedManagedRoot, removeOperationOwnedPaths } from './gitops/createCleanup';
|
||||
import { managedAreaBase } from './gitops/managedPaths';
|
||||
import type { GitHttpRequest, GitHttpResponse, HttpClient } from 'isomorphic-git/http/node';
|
||||
|
||||
// isomorphic-git is the heaviest dependency in the backend (~5 MB) and only
|
||||
// fires when a stack is created from a Git source. Lazy-load it so cold
|
||||
// boots without any Git-sourced stacks never parse the module.
|
||||
type IsomorphicGit = typeof import('isomorphic-git')['default'];
|
||||
type IsomorphicGitHttp = typeof import('isomorphic-git/http/node')['default'];
|
||||
|
||||
let cachedGit: IsomorphicGit | undefined;
|
||||
let cachedGitHttp: IsomorphicGitHttp | undefined;
|
||||
|
||||
async function loadIsomorphicGit(): Promise<{ git: IsomorphicGit; gitHttp: IsomorphicGitHttp }> {
|
||||
if (!cachedGit || !cachedGitHttp) {
|
||||
const [gitMod, gitHttpMod] = await Promise.all([
|
||||
import('isomorphic-git'),
|
||||
import('isomorphic-git/http/node'),
|
||||
]);
|
||||
cachedGit = gitMod.default;
|
||||
cachedGitHttp = gitHttpMod.default;
|
||||
}
|
||||
return { git: cachedGit, gitHttp: cachedGitHttp };
|
||||
}
|
||||
|
||||
function cloneTimeoutError(): Error & { code: string } {
|
||||
return Object.assign(new Error('Clone timed out'), { code: 'ETIMEDOUT' });
|
||||
}
|
||||
|
||||
async function collectGitBody(body: AsyncIterableIterator<Uint8Array>, signal: AbortSignal): Promise<Uint8Array> {
|
||||
const chunks: Uint8Array[] = [];
|
||||
let size = 0;
|
||||
for await (const chunk of body) {
|
||||
if (signal.aborted) throw cloneTimeoutError();
|
||||
chunks.push(chunk);
|
||||
size += chunk.byteLength;
|
||||
}
|
||||
const result = new Uint8Array(size);
|
||||
let offset = 0;
|
||||
for (const chunk of chunks) {
|
||||
result.set(chunk, offset);
|
||||
offset += chunk.byteLength;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function responseBodyIterator(body: ReadableStream<Uint8Array> | null): AsyncIterableIterator<Uint8Array> {
|
||||
async function* iterate(): AsyncIterableIterator<Uint8Array> {
|
||||
if (!body) return;
|
||||
const reader = body.getReader();
|
||||
try {
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) return;
|
||||
yield value;
|
||||
}
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
}
|
||||
return iterate();
|
||||
}
|
||||
|
||||
/**
|
||||
* Mutable counter shared across every request in one clone so the size cap
|
||||
* is cumulative, and so the caller can tell a size abort from a timeout via
|
||||
* `exceeded` rather than the thrown error (which isomorphic-git may wrap).
|
||||
*/
|
||||
interface CloneSizeState {
|
||||
exceeded: boolean;
|
||||
received: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Wrap a response-body iterator with a cumulative byte counter shared
|
||||
* across every request in one clone. When the running total crosses
|
||||
* `maxBytes`, flip `state.exceeded`, abort the transport (closing the
|
||||
* socket so the download stops), and throw to unwind the stream. The
|
||||
* caller distinguishes a size abort from a timeout/transport abort via
|
||||
* `state.exceeded` rather than the thrown error, which isomorphic-git may
|
||||
* wrap.
|
||||
*/
|
||||
export function countingBodyIterator(
|
||||
src: AsyncIterableIterator<Uint8Array>,
|
||||
controller: AbortController,
|
||||
maxBytes: number,
|
||||
state: CloneSizeState,
|
||||
): AsyncIterableIterator<Uint8Array> {
|
||||
async function* iterate(): AsyncIterableIterator<Uint8Array> {
|
||||
for await (const chunk of src) {
|
||||
state.received += chunk.byteLength;
|
||||
if (state.received > maxBytes) {
|
||||
state.exceeded = true;
|
||||
controller.abort();
|
||||
throw new Error('Clone exceeded the maximum allowed size');
|
||||
}
|
||||
yield chunk;
|
||||
}
|
||||
}
|
||||
return iterate();
|
||||
}
|
||||
|
||||
function createAbortableGitHttp(
|
||||
controller: AbortController,
|
||||
maxBytes: number,
|
||||
state: CloneSizeState,
|
||||
): HttpClient {
|
||||
const signal = controller.signal;
|
||||
return {
|
||||
async request(request: GitHttpRequest): Promise<GitHttpResponse> {
|
||||
if (signal.aborted) {
|
||||
throw cloneTimeoutError();
|
||||
}
|
||||
|
||||
const response = await fetch(request.url, {
|
||||
method: request.method ?? 'GET',
|
||||
headers: request.headers,
|
||||
body: request.body ? await collectGitBody(request.body, signal) : undefined,
|
||||
signal,
|
||||
});
|
||||
|
||||
return {
|
||||
url: response.url,
|
||||
method: request.method,
|
||||
statusCode: response.status,
|
||||
statusMessage: response.statusText,
|
||||
headers: Object.fromEntries(response.headers.entries()),
|
||||
body: countingBodyIterator(responseBodyIterator(response.body), controller, maxBytes, state),
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* GitSourceService - fetch compose files from a Git repository and apply
|
||||
@@ -362,17 +234,19 @@ const DEFAULT_FETCH_TIMEOUT_MS = 30_000;
|
||||
const TEMP_DIR_MAX_AGE_MS = 60 * 60 * 1000; // 1 hour
|
||||
const WEBHOOK_DEBOUNCE_MS = 10_000;
|
||||
|
||||
// Ceiling on how many bytes a single clone may download (the compressed pack
|
||||
// from the Git host) before it is aborted. This bounds network transfer and
|
||||
// abuse, not the decompressed on-disk checkout; it is paired with
|
||||
// MAX_REPO_FILE_BYTES and the 30s timeout. Generous default; operators with a
|
||||
// legitimately large monorepo can raise it via GITSOURCE_MAX_CLONE_BYTES.
|
||||
// Ceiling on the on-disk size of a single clone workspace before the fetch
|
||||
// is killed. Enforced by a watchdog that stats the workspace during the
|
||||
// clone (the native git transport streams the pack straight to disk, so
|
||||
// unlike the previous HTTP client there is no byte counter to hook). Paired
|
||||
// with MAX_REPO_FILE_BYTES and the timeout. Generous default; operators
|
||||
// with a legitimately large monorepo can raise it via GITSOURCE_MAX_CLONE_BYTES.
|
||||
const DEFAULT_MAX_CLONE_BYTES = 100 * 1024 * 1024; // 100 MB
|
||||
|
||||
// Per-file ceiling for the compose/env file read into memory after the clone.
|
||||
// These files are KB-scale in practice; the clone byte cap bounds the
|
||||
// compressed download, not the decompressed working tree, so this guards the
|
||||
// in-memory read against a single huge (or highly compressible) file.
|
||||
// These files are KB-scale in practice; the clone byte cap bounds the total
|
||||
// on-disk workspace, not any single file within it, so this guards the
|
||||
// in-memory read against one outsized file inside an otherwise in-budget
|
||||
// checkout.
|
||||
const MAX_REPO_FILE_BYTES = 10 * 1024 * 1024; // 10 MB
|
||||
|
||||
function maxCloneBytes(): number {
|
||||
@@ -391,10 +265,9 @@ function formatBytes(bytes: number): string {
|
||||
|
||||
/**
|
||||
* Remove any inline credentials and Authorization headers from an error
|
||||
* message before it lands in a log or an API response. isomorphic-git
|
||||
* tends to include the fetch URL in thrown errors; if a PAT ever leaks
|
||||
* into that URL (we try to avoid it via `onAuth`, but be defensive),
|
||||
* strip it here.
|
||||
* message before it lands in a log or an API response. Git errors tend to
|
||||
* include the fetch URL; if a PAT ever leaks into a URL (we never send one,
|
||||
* but be defensive), strip it here.
|
||||
*/
|
||||
function scrubCredentials(message: string): string {
|
||||
return message
|
||||
@@ -452,63 +325,10 @@ export function repoHost(url: string): string {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Node's global `fetch()` reports every transport-level failure as a bare
|
||||
* `TypeError('fetch failed')` and hides the real reason on `error.cause`
|
||||
* (occasionally nested one level deeper through undici). Walk the cause
|
||||
* chain for the first Node error code so DNS / connection / TLS failures
|
||||
* can be translated into an actionable message instead of "fetch failed",
|
||||
* which reads like an internal Sencho bug.
|
||||
*/
|
||||
function findCauseCode(err: unknown): { code?: string } {
|
||||
let cur: unknown = err;
|
||||
for (let depth = 0; depth < 5 && cur; depth++) {
|
||||
const code = (cur as { code?: unknown }).code;
|
||||
if (typeof code === 'string' && code) {
|
||||
return { code };
|
||||
}
|
||||
cur = (cur as { cause?: unknown }).cause;
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
/**
|
||||
* Translate a Node transport error code into a GitSourceError with a
|
||||
* host-qualified, user-actionable message. Returns null for codes we do
|
||||
* not specifically recognise so the caller can fall through to its generic
|
||||
* handling. `host` is the bare hostname from `repoHost()` (never carries a
|
||||
* credential), so it is safe to surface.
|
||||
*/
|
||||
function transportError(code: string, host: string): GitSourceError | null {
|
||||
const dest = host && host !== 'unknown' ? ` ${host}` : ' the repository host';
|
||||
switch (code) {
|
||||
case 'ENOTFOUND':
|
||||
case 'EAI_AGAIN':
|
||||
return new GitSourceError('NETWORK_TIMEOUT', `Could not resolve${dest}. Check the repository URL and your network or DNS.`);
|
||||
case 'ECONNREFUSED':
|
||||
return new GitSourceError('NETWORK_TIMEOUT', `Connection refused by${dest}.`);
|
||||
case 'ECONNRESET':
|
||||
return new GitSourceError('NETWORK_TIMEOUT', `Connection to${dest} was reset. Retry; if it persists, check the host.`);
|
||||
case 'ETIMEDOUT':
|
||||
case 'UND_ERR_CONNECT_TIMEOUT':
|
||||
case 'UND_ERR_HEADERS_TIMEOUT':
|
||||
case 'UND_ERR_BODY_TIMEOUT':
|
||||
return new GitSourceError('NETWORK_TIMEOUT', `Timed out reaching${dest}.`);
|
||||
case 'DEPTH_ZERO_SELF_SIGNED_CERT':
|
||||
case 'SELF_SIGNED_CERT_IN_CHAIN':
|
||||
case 'UNABLE_TO_VERIFY_LEAF_SIGNATURE':
|
||||
case 'CERT_HAS_EXPIRED':
|
||||
case 'ERR_TLS_CERT_ALTNAME_INVALID':
|
||||
return new GitSourceError('GIT_ERROR', `TLS certificate error reaching${dest} (${code}). The host certificate could not be verified.`);
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Git LFS stores large files as small pointer stubs in the working tree.
|
||||
* The pointer is a short text file that always begins with this line.
|
||||
* isomorphic-git does not resolve LFS, so if the compose or env file is
|
||||
* The fetch does not resolve LFS, so if the compose or env file is
|
||||
* tracked through LFS we would silently write the pointer as content.
|
||||
* Detect this and refuse, with a clear error, before it ever lands on
|
||||
* disk.
|
||||
@@ -524,7 +344,7 @@ export function isLfsPointer(content: string): boolean {
|
||||
|
||||
/**
|
||||
* Check whether the cloned tree references Git submodules. We do not
|
||||
* fetch submodule contents (isomorphic-git does not support them), so
|
||||
* fetch submodule contents (clones run with --no-recurse-submodules), so
|
||||
* warn the caller that any paths inside submodule directories will be
|
||||
* empty at deploy time.
|
||||
*/
|
||||
@@ -557,8 +377,8 @@ async function readRepoFile(rootDir: string, relPath: string, label: string): Pr
|
||||
if (stat.isSymbolicLink()) {
|
||||
throw new GitSourceError('FILE_NOT_FOUND', `${label} cannot be a symbolic link.`);
|
||||
}
|
||||
// Bound the in-memory read. The clone byte cap only limits the compressed
|
||||
// download; a single decompressed file can still be large, so reject an
|
||||
// Bound the in-memory read. The clone byte cap only limits the total
|
||||
// on-disk workspace, not any single file within it, so reject an
|
||||
// oversized compose/env file before reading it into a string.
|
||||
if (stat.size > MAX_REPO_FILE_BYTES) {
|
||||
throw new GitSourceError('GIT_ERROR', `${label} is too large (${formatBytes(stat.size)}); the maximum is ${formatBytes(MAX_REPO_FILE_BYTES)}.`);
|
||||
@@ -1087,10 +907,14 @@ export class GitSourceService {
|
||||
// ─── Fetch ───────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Clone a repo into a throwaway temp dir, run `fn` against the checkout, and
|
||||
* always clean up. Centralizes the clone timeout, size cap, commit-sha read,
|
||||
* and submodule warning so both fetchFromGit (reads compose/env files) and
|
||||
* listRepoTree (lists the working tree) share one hardened clone path.
|
||||
* Resolve the configured branch to an immutable commit, clone exactly that
|
||||
* snapshot into a throwaway workspace, run `fn` against the checkout, and
|
||||
* always clean up. Centralizes resolution, the fetch timeout, the size
|
||||
* watchdog, commit verification, and the submodule warning so both
|
||||
* fetchFromGit (reads compose/env files) and listRepoTree (lists the
|
||||
* working tree) share one hardened path. Transport mechanics live in
|
||||
* `./git/nativeGitTransport`; failures arrive pre-classified or as
|
||||
* structured transport failures mapped below.
|
||||
*/
|
||||
private async withClonedRepo<T>(
|
||||
params: { repoUrl: string; branch: string; token?: string | null; timeoutMs?: number },
|
||||
@@ -1098,76 +922,55 @@ export class GitSourceService {
|
||||
): Promise<T> {
|
||||
const { repoUrl, branch, token } = params;
|
||||
const timeoutMs = params.timeoutMs ?? DEFAULT_FETCH_TIMEOUT_MS;
|
||||
const dir = await createTempDir();
|
||||
|
||||
// isomorphic-git's onAuth callback hands credentials to the HTTP layer
|
||||
// without them touching the URL string, keeping tokens out of any error
|
||||
// messages generated during the clone.
|
||||
const onAuth = token
|
||||
? () => ({ username: 'x-access-token', password: token })
|
||||
: undefined;
|
||||
const root = await createTempDir();
|
||||
|
||||
try {
|
||||
const { git } = await loadIsomorphicGit();
|
||||
// Bound clone duration and abort the HTTP transport so timed-out
|
||||
// fetches do not keep sockets and packfile streams alive.
|
||||
let timer: NodeJS.Timeout | undefined;
|
||||
const controller = new AbortController();
|
||||
const maxBytes = maxCloneBytes();
|
||||
const sizeState = { exceeded: false, received: 0 };
|
||||
const timeout = new Promise<never>((_, reject) => {
|
||||
timer = setTimeout(() => {
|
||||
controller.abort();
|
||||
reject(cloneTimeoutError());
|
||||
}, timeoutMs);
|
||||
const resolved = await nativeGitTransport.resolveRef({
|
||||
repoUrl,
|
||||
ref: branch,
|
||||
token,
|
||||
timeoutMs,
|
||||
workspaceRoot: root,
|
||||
});
|
||||
const fetched = await nativeGitTransport.fetchAtCommit({
|
||||
repoUrl,
|
||||
ref: branch,
|
||||
token,
|
||||
timeoutMs,
|
||||
commitSha: resolved.commitSha,
|
||||
workspaceRoot: root,
|
||||
maxBytes: maxCloneBytes(),
|
||||
});
|
||||
try {
|
||||
await Promise.race([
|
||||
git.clone({
|
||||
fs: { promises: fsPromises },
|
||||
http: createAbortableGitHttp(controller, maxBytes, sizeState),
|
||||
dir,
|
||||
url: repoUrl,
|
||||
ref: branch,
|
||||
singleBranch: true,
|
||||
depth: 1,
|
||||
noTags: true,
|
||||
onAuth,
|
||||
}),
|
||||
timeout,
|
||||
]);
|
||||
} catch (e) {
|
||||
// A size abort surfaces as a generic transport error once
|
||||
// isomorphic-git unwinds, so detect it via the shared flag.
|
||||
if (sizeState.exceeded) {
|
||||
throw new GitSourceError(
|
||||
'GIT_ERROR',
|
||||
`Repository exceeds the maximum clone size of ${formatBytes(maxBytes)}.`,
|
||||
);
|
||||
}
|
||||
throw this.mapGitError(e as Error, Boolean(token), repoHost(repoUrl));
|
||||
} finally {
|
||||
if (timer) clearTimeout(timer);
|
||||
}
|
||||
|
||||
const log = await git.log({ fs: { promises: fsPromises }, dir, ref: branch, depth: 1 });
|
||||
if (!log.length) {
|
||||
throw new GitSourceError('GIT_ERROR', 'Repository has no commits on the requested branch.');
|
||||
}
|
||||
const commitSha = log[0].oid;
|
||||
|
||||
// Submodule detection: non-fatal, surfaced as a warning. isomorphic-git
|
||||
// does not recursively clone submodules, so any path that lives inside
|
||||
// a submodule directory will be empty after apply. Users need to know.
|
||||
// Submodule detection: non-fatal, surfaced as a warning. Clones run
|
||||
// with --no-recurse-submodules, so any path that lives inside a
|
||||
// submodule directory will be empty after apply. Users need to know.
|
||||
const warnings: string[] = [];
|
||||
if (await hasSubmodules(dir)) {
|
||||
if (await hasSubmodules(fetched.dir)) {
|
||||
console.warn(`[GitSource] Submodules detected in ${repoHost(repoUrl)}; contents not cloned.`);
|
||||
warnings.push(SUBMODULE_WARNING);
|
||||
}
|
||||
|
||||
return await fn(dir, commitSha, warnings);
|
||||
return await fn(fetched.dir, fetched.commitSha, warnings);
|
||||
} catch (e) {
|
||||
if (isTransportFailure(e)) {
|
||||
// The classified message operators see is deliberately
|
||||
// sanitized and may be generic (unrecognized stderr); always
|
||||
// keep the raw reason and scrubbed stderr tail in the server
|
||||
// log so new git wording is diagnosable.
|
||||
const detail = scrubCredentials(
|
||||
`reason=${e.reason} exit=${'exitCode' in e ? e.exitCode : '-'} stderr=${('stderr' in e && e.stderr ? e.stderr : '').slice(-600)}`,
|
||||
);
|
||||
console.error(`[GitSource:transport] host=${sanitizeForLog(e.host)} ${detail}`);
|
||||
if (isDebugEnabled() && 'argv' in e && e.argv?.length) {
|
||||
console.error(`[GitSource:transport] argv=[${e.argv.map((a) => sanitizeForLog(a)).join(' ')}]`);
|
||||
}
|
||||
const classified = classifyGitFailure(e);
|
||||
throw new GitSourceError(classified.code, classified.message);
|
||||
}
|
||||
throw e;
|
||||
} finally {
|
||||
await removeTempDir(dir);
|
||||
await removeTempDir(root);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1297,72 +1100,6 @@ export class GitSourceService {
|
||||
return { files, truncated };
|
||||
}
|
||||
|
||||
private mapGitError(err: Error, hasToken: boolean, host = 'unknown'): GitSourceError {
|
||||
const raw = scrubCredentials(err.message || String(err));
|
||||
const code = (err as Error & { code?: string }).code;
|
||||
// isomorphic-git's HttpError exposes the numeric status on .data; inspect
|
||||
// it directly so a 404 is not misclassified as auth failure. GitHub hides
|
||||
// private-repo existence by returning 404 to unauthenticated requests, so
|
||||
// we also treat 401/403 without a supplied token as "not found or private"
|
||||
// to guide the user to add a token rather than "check your token" when
|
||||
// they never provided one.
|
||||
const statusCode = (err as Error & { data?: { statusCode?: number } }).data?.statusCode;
|
||||
|
||||
// GitHub returns 404 for both "repo genuinely missing" and "private repo
|
||||
// the caller cannot see". We cannot distinguish the two without a second
|
||||
// probe, so tailor the hint by whether credentials were supplied:
|
||||
// - no token: suggest adding one for private repos
|
||||
// - token present: suggest checking the URL and token scopes, since a
|
||||
// valid token against a missing or wrong-scoped repo also lands here
|
||||
if (statusCode === 404) {
|
||||
if (hasToken) {
|
||||
return new GitSourceError('REPO_NOT_FOUND', 'Repository not found. Verify the URL and that your token has read access to this repo.');
|
||||
}
|
||||
return new GitSourceError('REPO_NOT_FOUND', 'Repository not found, or it is private. Add a Personal Access Token if the repo is private.');
|
||||
}
|
||||
if (statusCode === 401 || statusCode === 403) {
|
||||
if (hasToken) {
|
||||
return new GitSourceError('AUTH_FAILED', 'Repository authentication failed. Check your token.');
|
||||
}
|
||||
return new GitSourceError('REPO_NOT_FOUND', 'Repository not found, or it is private. Add a Personal Access Token if the repo is private.');
|
||||
}
|
||||
|
||||
// Transport failures: Node's fetch() throws a bare "fetch failed"
|
||||
// TypeError with the real reason (ENOTFOUND, ECONNREFUSED, TLS, ...)
|
||||
// on err.cause. Translate the underlying code before falling through
|
||||
// to the generic branches, which only see the useless "fetch failed".
|
||||
if (!statusCode) {
|
||||
const cause = findCauseCode(err);
|
||||
if (cause.code) {
|
||||
const mapped = transportError(cause.code, host);
|
||||
if (mapped) return mapped;
|
||||
}
|
||||
}
|
||||
|
||||
// Fallbacks for errors without a numeric status attached (e.g. git CLI
|
||||
// output, DNS/lib errors, or future isomorphic-git transports that do
|
||||
// not populate err.data.statusCode). Kept as defense-in-depth.
|
||||
if (/401|403|authentication/i.test(raw)) {
|
||||
return hasToken
|
||||
? new GitSourceError('AUTH_FAILED', 'Repository authentication failed. Check your token.')
|
||||
: new GitSourceError('REPO_NOT_FOUND', 'Repository not found, or it is private. Add a Personal Access Token if the repo is private.');
|
||||
}
|
||||
if (code === 'NotFoundError' || /404|not found|could not resolve/i.test(raw)) {
|
||||
return new GitSourceError('REPO_NOT_FOUND', 'Repository not found or not accessible.');
|
||||
}
|
||||
if (code === 'ResolveRefError' || /resolve ref|unknown ref|couldn't find remote ref|reference not found/i.test(raw)) {
|
||||
return new GitSourceError('BRANCH_NOT_FOUND', 'Branch not found in the repository.');
|
||||
}
|
||||
if (code === 'ECONNABORTED' || /timeout|timed out|ETIMEDOUT|ENOTFOUND|ECONNREFUSED/i.test(raw)) {
|
||||
return new GitSourceError('NETWORK_TIMEOUT', 'Network timeout or host unreachable.');
|
||||
}
|
||||
// Last-resort: an HttpError with a status we did not specifically handle.
|
||||
if (code === 'HttpError') {
|
||||
return new GitSourceError('GIT_ERROR', `Unexpected HTTP response from git host${statusCode ? ` (${statusCode})` : ''}.`);
|
||||
}
|
||||
return new GitSourceError('GIT_ERROR', raw);
|
||||
}
|
||||
|
||||
// ─── Validation ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
|
||||
/**
|
||||
* Credential handoff for authenticated HTTPS clones.
|
||||
*
|
||||
* The token must never appear in argv, in the repository URL, or in any
|
||||
* generated subprocess metadata. Git's credential-helper protocol is the
|
||||
* sanctioned side channel: git invokes the helper with request attributes on
|
||||
* stdin and reads key=value lines from stdout. We write a tiny per-fetch
|
||||
* helper script that echoes the password straight from an environment
|
||||
* variable, and pass the actual token only through the child process env.
|
||||
*
|
||||
* The helper lives under the fetch workspace's `.meta/` directory, which the
|
||||
* caller deletes in its `finally` block together with the rest of the
|
||||
* workspace.
|
||||
*
|
||||
* Locating the helper: its path travels in the child environment, never in
|
||||
* the config value. Git treats `credential.helper` as a SHELL string, not as
|
||||
* argv: run-command's shell detection routes the value through `sh -c` as
|
||||
* soon as it contains any of `|&;<>()$\`\\"' \t\n*?[#~=%`, and a space is in
|
||||
* that set. Interpolating the path directly therefore word-splits whenever
|
||||
* the workspace sits under a directory with a space in its name (a plain
|
||||
* `/tmp/some dir` or a Windows `C:/Users/Ada Lovelace/AppData/Local/Temp`),
|
||||
* and git ends up executing the first path segment. Quoting the interpolated
|
||||
* path fixes that one case but keeps the path inside a shell string, one
|
||||
* unusual character away from breaking again. Pointing the shell at an
|
||||
* environment variable instead makes the config value a CONSTANT, so no
|
||||
* workspace path can affect how it parses.
|
||||
*/
|
||||
|
||||
export const GIT_TOKEN_ENV_VAR = 'SENCHO_GIT_TOKEN';
|
||||
export const GIT_HELPER_PATH_ENV_VAR = 'SENCHO_GIT_HELPER';
|
||||
export const GIT_HELPER_USERNAME = 'x-access-token';
|
||||
|
||||
/**
|
||||
* The `credential.helper` value. The leading `!` tells git to run the rest as
|
||||
* a shell command (gitcredentials(7)); the quoted variable expands inside
|
||||
* that shell to the path we exported, whatever characters it holds.
|
||||
*/
|
||||
export const CREDENTIAL_HELPER_CONFIG_VALUE = `!"$${GIT_HELPER_PATH_ENV_VAR}"`;
|
||||
|
||||
/**
|
||||
* One POSIX script on every platform. Because the value above always routes
|
||||
* through a shell, the helper is always launched by git's own `sh` (Git for
|
||||
* Windows ships one and uses it for exactly this), so a `.cmd` variant would
|
||||
* add a second dialect without ever being reached more directly.
|
||||
*/
|
||||
const HELPER_SCRIPT = '#!/bin/sh\n'
|
||||
+ `printf 'username=${GIT_HELPER_USERNAME}\\n'\n`
|
||||
+ `printf 'password=%s\\n' "$${GIT_TOKEN_ENV_VAR}"\n`;
|
||||
|
||||
/** Render the helper script body; exported for tests pinning the no-secret invariant. */
|
||||
export function renderCredentialHelper(): string {
|
||||
return HELPER_SCRIPT;
|
||||
}
|
||||
|
||||
/**
|
||||
* Write the helper executable into `metaDir` and return its absolute path
|
||||
* (forward slashes; git's shell accepts that spelling on Windows too). The
|
||||
* script contains only a variable REFERENCE, never the secret itself.
|
||||
*/
|
||||
export async function writeCredentialHelper(metaDir: string): Promise<string> {
|
||||
const helperPath = path.join(metaDir, 'credential-helper.sh');
|
||||
await fs.writeFile(helperPath, HELPER_SCRIPT, { mode: 0o700 });
|
||||
return helperPath.split(path.sep).join('/');
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
/**
|
||||
* Classification of native-git failures into the public GitSourceErrorCode
|
||||
* contract.
|
||||
*
|
||||
* Deliberately class-free: this module returns plain {code, message} data and
|
||||
* never imports GitSourceService, so the dependency graph stays one-way
|
||||
* (service -> git/*) and the classifier is unit-testable in isolation. The
|
||||
* service wraps the returned pair in its own GitSourceError.
|
||||
*
|
||||
* Two behaviors are contractual and pinned by tests; do not change them:
|
||||
* 1. Authentication failure WITH a supplied token reports AUTH_FAILED,
|
||||
* which the HTTP layer maps to 400, never 401, because the frontend's
|
||||
* global logout trips on any API-level 401.
|
||||
* 2. A 401/403-shaped refusal WITHOUT a token reports REPO_NOT_FOUND with
|
||||
* a private-repo hint, mirroring GitHub's masking of private repos.
|
||||
*/
|
||||
|
||||
export type TransportFacingCode =
|
||||
| 'REPO_NOT_FOUND'
|
||||
| 'AUTH_FAILED'
|
||||
| 'BRANCH_NOT_FOUND'
|
||||
| 'NETWORK_TIMEOUT'
|
||||
| 'GIT_ERROR';
|
||||
|
||||
/** Structured failure raised by the native transport; classified below. */
|
||||
export type TransportFailureReason =
|
||||
| 'invalid-url'
|
||||
| 'invalid-ref'
|
||||
| 'git-missing'
|
||||
| 'git-old'
|
||||
| 'ref-not-found'
|
||||
| 'tip-changed'
|
||||
| 'size'
|
||||
| 'timeout'
|
||||
| 'exit';
|
||||
|
||||
interface TransportFailureBase {
|
||||
/** Branded discriminant so isTransportFailure cannot false-positive on foreign errors. */
|
||||
readonly transportFailure: true;
|
||||
host: string;
|
||||
hasToken: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Discriminated on `reason`: each variant carries exactly the payload its
|
||||
* classifier branch needs (e.g. `size` must always know `maxBytes`, so the
|
||||
* operator-facing breach message can never render "0 B").
|
||||
*/
|
||||
export type TransportFailure = TransportFailureBase & (
|
||||
| { reason: 'invalid-url' }
|
||||
| { reason: 'invalid-ref' }
|
||||
| { reason: 'git-missing'; stderr?: string }
|
||||
| { reason: 'git-old'; stderr?: string }
|
||||
| { reason: 'ref-not-found' }
|
||||
| { reason: 'tip-changed' }
|
||||
| { reason: 'size'; maxBytes: number }
|
||||
| { reason: 'timeout' }
|
||||
| { reason: 'exit'; stderr?: string; exitCode?: number; /** Full child argv, attached for debug diagnostics only. */ argv?: string[] }
|
||||
);
|
||||
|
||||
/**
|
||||
* Defensive redaction mirroring GitSourceService.scrubCredentials. Kept local
|
||||
* instead of imported to preserve the one-way dependency direction; git never
|
||||
* receives credentials via URL, so this only guards against operators pasting
|
||||
* user:pass@ URLs that servers echo back in error text.
|
||||
*/
|
||||
function redactCredentials(text: string): string {
|
||||
return text
|
||||
.replace(/https?:\/\/[^/\s:@]+:[^/\s@]+@/gi, 'https://***:***@')
|
||||
.replace(/(authorization[:=]\s*)[^\s,;]+/gi, '$1***')
|
||||
.replace(/(token[:=]\s*)[^\s,;]+/gi, '$1***')
|
||||
.replace(/(password[:=]\s*)[^\s,;]+/gi, '$1***');
|
||||
}
|
||||
|
||||
function formatBytes(bytes: number): string {
|
||||
if (bytes >= 1024 * 1024) return `${Math.round(bytes / (1024 * 1024))} MB`;
|
||||
if (bytes >= 1024) return `${Math.round(bytes / 1024)} KB`;
|
||||
return `${bytes} B`;
|
||||
}
|
||||
|
||||
function hostQualifier(host: string): string {
|
||||
return host && host !== 'unknown' ? ` ${host}` : ' the repository host';
|
||||
}
|
||||
|
||||
/** Shared hint for refusals where a private repo and a missing one are indistinguishable. */
|
||||
const PRIVATE_REPO_HINT = 'Repository not found, or it is private. Add a Personal Access Token if the repo is private.';
|
||||
|
||||
/** Last few stderr lines, scrubbed, for the generic GIT_ERROR fallback. */
|
||||
function stderrTail(stderr: string | undefined): string {
|
||||
if (!stderr) return '';
|
||||
const lines = redactCredentials(stderr).trim().split(/\r?\n/).filter(Boolean);
|
||||
return lines.slice(-3).join(' ').slice(0, 400);
|
||||
}
|
||||
|
||||
export function classifyGitFailure(
|
||||
failure: TransportFailure,
|
||||
): { code: TransportFacingCode; message: string } {
|
||||
const dest = hostQualifier(failure.host);
|
||||
|
||||
// Structured outcomes decided by the transport itself, before any
|
||||
// stderr guessing.
|
||||
switch (failure.reason) {
|
||||
case 'invalid-url':
|
||||
return { code: 'GIT_ERROR', message: 'Unsupported repository URL. Use an https:// URL without embedded credentials.' };
|
||||
case 'invalid-ref':
|
||||
return { code: 'GIT_ERROR', message: 'Unsupported branch name. Use the branch name as the remote reports it.' };
|
||||
case 'git-missing':
|
||||
return { code: 'GIT_ERROR', message: failure.stderr || 'The git command was not found on PATH.' };
|
||||
case 'git-old':
|
||||
return { code: 'GIT_ERROR', message: failure.stderr || 'The installed git client is too old.' };
|
||||
case 'ref-not-found':
|
||||
return { code: 'BRANCH_NOT_FOUND', message: 'Branch not found in the repository.' };
|
||||
case 'tip-changed':
|
||||
return { code: 'GIT_ERROR', message: 'Repository tip changed during fetch; retry the pull.' };
|
||||
case 'size':
|
||||
return {
|
||||
code: 'GIT_ERROR',
|
||||
message: `Repository exceeds the maximum clone size of ${formatBytes(failure.maxBytes)}.`,
|
||||
};
|
||||
case 'timeout':
|
||||
return { code: 'NETWORK_TIMEOUT', message: `Timed out reaching${dest}.` };
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
const raw = redactCredentials((failure.stderr ?? '').toLowerCase());
|
||||
|
||||
// Auth-shaped refusals. Native git phrases these two ways: with a token
|
||||
// it gets "Authentication failed for '<url>'"; without one it cannot even
|
||||
// answer and reports the disabled terminal prompt.
|
||||
if (/could not read username|could not read password/.test(raw)) {
|
||||
// Prompting was suppressed, so the host refused the credentials it
|
||||
// was given (possibly none): mask like GitHub hides private repos.
|
||||
return {
|
||||
code: 'REPO_NOT_FOUND',
|
||||
message: PRIVATE_REPO_HINT,
|
||||
};
|
||||
}
|
||||
if (/authentication failed|\b40[13]\b/.test(raw)) {
|
||||
return failure.hasToken
|
||||
? { code: 'AUTH_FAILED', message: 'Repository authentication failed. Check your token.' }
|
||||
: {
|
||||
code: 'REPO_NOT_FOUND',
|
||||
message: PRIVATE_REPO_HINT,
|
||||
};
|
||||
}
|
||||
if (/remote branch .+ not found in upstream|branch not found/.test(raw)) {
|
||||
return { code: 'BRANCH_NOT_FOUND', message: 'Branch not found in the repository.' };
|
||||
}
|
||||
if (/repository[\s\S]*\bnot found\b|not found in upstream/.test(raw)) {
|
||||
return {
|
||||
code: 'REPO_NOT_FOUND',
|
||||
message: failure.hasToken
|
||||
? 'Repository not found. Verify the URL and that your token has read access to this repo.'
|
||||
: PRIVATE_REPO_HINT,
|
||||
};
|
||||
}
|
||||
|
||||
// TLS failures before generic network wording, so certificate problems do
|
||||
// not read as connectivity problems.
|
||||
if (/ssl certificate problem|server certificate verification failed|certificate subject name|unable to get local issuer certificate|self[- ]signed certificate/.test(raw)) {
|
||||
return { code: 'GIT_ERROR', message: `TLS certificate error reaching${dest}. The host certificate could not be verified.` };
|
||||
}
|
||||
|
||||
// Network family.
|
||||
if (/could not resolve host|name or service not known|temporary failure in name resolution/.test(raw)) {
|
||||
return { code: 'NETWORK_TIMEOUT', message: `Could not resolve${dest}. Check the repository URL and your network or DNS.` };
|
||||
}
|
||||
if (/connection refused|could not connect to server/.test(raw)) {
|
||||
return { code: 'NETWORK_TIMEOUT', message: `Connection refused by${dest}.` };
|
||||
}
|
||||
if (/connection timed out|operation timed out|connection was reset|remote end hung up|connection reset by peer/.test(raw)) {
|
||||
return { code: 'NETWORK_TIMEOUT', message: `Connection to${dest} failed. Retry; if it persists, check the host or your network.` };
|
||||
}
|
||||
|
||||
const tail = stderrTail(failure.stderr);
|
||||
return { code: 'GIT_ERROR', message: tail ? `Git fetch failed: ${tail}` : 'Git fetch failed.' };
|
||||
}
|
||||
|
||||
/**
|
||||
* Structural type guard. The branded `transportFailure` discriminant makes
|
||||
* false positives impossible: arbitrary errors that happen to carry
|
||||
* reason/host fields are not mistaken for transport failures at the service
|
||||
* boundary.
|
||||
*/
|
||||
export function isTransportFailure(e: unknown): e is TransportFailure {
|
||||
return typeof e === 'object' && e !== null && (e as { transportFailure?: unknown }).transportFailure === true;
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
import { execFile } from 'child_process';
|
||||
|
||||
/**
|
||||
* Locate and qualify the git binary once per process. Git Sources requires
|
||||
* the native git CLI at runtime; Docker images install it explicitly, but
|
||||
* bare-metal and `npm run dev` hosts may not have it, so the probe produces
|
||||
* an actionable failure instead of a confusing ENOENT deep inside a clone.
|
||||
*/
|
||||
|
||||
// Shallow single-branch clone and ls-remote behaviors used here are stable
|
||||
// long before this; the floor mainly guards against ancient builds with
|
||||
// different stderr wording.
|
||||
const MIN_GIT_VERSION = [2, 40, 0] as const;
|
||||
|
||||
let cachedProbe: Promise<string> | undefined;
|
||||
|
||||
function runGitVersion(): Promise<string> {
|
||||
return new Promise((resolve, reject) => {
|
||||
execFile('git', ['--version'], { windowsHide: true, timeout: 10_000 }, (err, stdout) => {
|
||||
if (err) {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
resolve(stdout.trim());
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function parseVersion(output: string): number[] | null {
|
||||
const match = /git version (\d+)\.(\d+)(?:\.(\d+))?/.exec(output);
|
||||
if (!match) return null;
|
||||
return [Number(match[1]), Number(match[2]), Number(match[3] ?? 0)];
|
||||
}
|
||||
|
||||
function versionAtLeast(actual: number[], min: readonly number[]): boolean {
|
||||
for (let i = 0; i < min.length; i++) {
|
||||
const a = actual[i] ?? 0;
|
||||
if (a > min[i]) return true;
|
||||
if (a < min[i]) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Probe `git --version` once and cache the result. Resolves with the raw
|
||||
* version output; rejects with an operator-actionable error when the binary
|
||||
* is missing or older than the supported floor.
|
||||
*/
|
||||
export function ensureGitBinary(): Promise<string> {
|
||||
cachedProbe ??= (async () => {
|
||||
let output: string;
|
||||
try {
|
||||
output = await runGitVersion();
|
||||
} catch (e) {
|
||||
cachedProbe = undefined;
|
||||
const cause = e instanceof Error ? ` (${e.message})` : '';
|
||||
throw new Error(
|
||||
'The git command could not be executed. Sencho requires the native git client for Git Sources. '
|
||||
+ 'If git is installed, check its permissions; otherwise install it (Docker images already include it) and restart.'
|
||||
+ cause,
|
||||
);
|
||||
}
|
||||
const parsed = parseVersion(output);
|
||||
if (!parsed || !versionAtLeast(parsed, MIN_GIT_VERSION)) {
|
||||
cachedProbe = undefined;
|
||||
throw new Error(
|
||||
`The installed git client is too old (${output || 'unrecognized'}). `
|
||||
+ `Git Sources requires git ${MIN_GIT_VERSION.join('.')} or newer.`,
|
||||
);
|
||||
}
|
||||
return output;
|
||||
})();
|
||||
return cachedProbe;
|
||||
}
|
||||
|
||||
let cachedExecPath: Promise<string> | undefined;
|
||||
|
||||
/**
|
||||
* Absolute path of git's exec directory (cached). Used on Windows to locate
|
||||
* the installation's bundled CA bundle, whose normal discovery goes through
|
||||
* system gitconfig that this transport deliberately strips.
|
||||
*/
|
||||
export function getGitExecPath(): Promise<string> {
|
||||
cachedExecPath ??= new Promise<string>((resolve, reject) => {
|
||||
execFile('git', ['--exec-path'], { windowsHide: true, timeout: 10_000 }, (err, stdout) => {
|
||||
if (err) {
|
||||
// Mirror ensureGitBinary: a transient failure must not poison
|
||||
// the cache for the lifetime of the process.
|
||||
cachedExecPath = undefined;
|
||||
reject(err);
|
||||
} else {
|
||||
resolve(stdout.trim());
|
||||
}
|
||||
});
|
||||
});
|
||||
return cachedExecPath;
|
||||
}
|
||||
|
||||
/** Test seam: forget the cached probes so subsequent calls re-run them. */
|
||||
export function resetGitBinaryProbeForTests(): void {
|
||||
cachedProbe = undefined;
|
||||
cachedExecPath = undefined;
|
||||
}
|
||||
@@ -0,0 +1,762 @@
|
||||
import { spawn, type ChildProcess } from 'child_process';
|
||||
import { promises as fs, existsSync } from 'fs';
|
||||
import os from 'os';
|
||||
import path from 'path';
|
||||
import { ensureGitBinary, getGitExecPath } from './gitBinary';
|
||||
import {
|
||||
CREDENTIAL_HELPER_CONFIG_VALUE,
|
||||
GIT_HELPER_PATH_ENV_VAR,
|
||||
GIT_TOKEN_ENV_VAR,
|
||||
writeCredentialHelper,
|
||||
} from './credentialHelper';
|
||||
import { isTransportFailure, type TransportFailure } from './errors';
|
||||
import type { FetchRequest, FetchResult, GitTransport, ResolveRequest } from './types';
|
||||
|
||||
/**
|
||||
* Native git transport: every Git operation is an `execFile`-style spawn of
|
||||
* the real git CLI with an argv array (never a shell), a hardened
|
||||
* environment, and per-invocation config flags.
|
||||
*
|
||||
* Hardening applied to every invocation:
|
||||
* - `GIT_CONFIG_NOSYSTEM=1` plus an isolated empty HOME/USERPROFILE so the
|
||||
* operator's ~/.gitconfig (credential helpers, insteadOf rewrites, hooks)
|
||||
* cannot influence fetches.
|
||||
* - `protocol.allow=never` with only https re-enabled: no file://, git://,
|
||||
* ext::, or ssh:// this early in the program.
|
||||
* - `core.hooksPath` pointed at an empty directory we own, so repository
|
||||
* scripts can never run. (A literal /dev/null works on Linux but not
|
||||
* Windows; an empty dir is portable.)
|
||||
* - `GIT_TERMINAL_PROMPT=0` and a neutralized GIT_ASKPASS so a missing or
|
||||
* wrong credential fails fast instead of hanging on a prompt.
|
||||
* - Every remaining git-config channel is pinned empty (GIT_CONFIG_GLOBAL /
|
||||
* GIT_CONFIG_SYSTEM to the null device, XDG_CONFIG_HOME cleared,
|
||||
* GIT_CONFIG_COUNT zeroed) and inherited GIT_TRACE is cleared so packet
|
||||
* dumps cannot carry URL material.
|
||||
* - The token reaches git ONLY through the credential helper reading
|
||||
* SENCHO_GIT_TOKEN from the child env; it never appears in argv or URLs.
|
||||
*
|
||||
* Dev/E2E certificate bridge: when NODE_EXTRA_CA_CERTS is set (the existing
|
||||
* dev/CI wiring for the e2e TLS fixture server), its CAs are combined with
|
||||
* platform defaults into <workspace>/.meta/combined-ca.pem and passed as
|
||||
* http.sslCAInfo, mirroring Node's add-not-replace semantics. Without the
|
||||
* variable, POSIX passes nothing (OpenSSL uses system trust) and Windows
|
||||
* pins Git's own bundled bundle (see detectWindowsCABundle).
|
||||
*/
|
||||
|
||||
const DEFAULT_TIMEOUT_MS = 30_000;
|
||||
const LS_REMOTE_MAX_MS = 10_000;
|
||||
const STDERR_CAP = 16_384;
|
||||
const WATCHDOG_INTERVAL_MS = 1_000;
|
||||
const SHA_PATTERN = /^[0-9a-f]{40}$|^[0-9a-f]{64}$/i;
|
||||
|
||||
interface RunResult {
|
||||
stdout: string;
|
||||
stderr: string;
|
||||
exitCode: number;
|
||||
}
|
||||
|
||||
interface RunOptions {
|
||||
cwd?: string;
|
||||
env: NodeJS.ProcessEnv;
|
||||
timeoutMs: number;
|
||||
onSpawn?: (child: ChildProcess) => void;
|
||||
}
|
||||
|
||||
function isTimeoutError(e: unknown): boolean {
|
||||
return typeof e === 'object' && e !== null && (e as { gitTimedOut?: unknown }).gitTimedOut === true;
|
||||
}
|
||||
|
||||
/** Flag an error as a git timeout so `isTimeoutError` recognises it downstream. */
|
||||
function asTimeoutError<T extends Error>(err: T): T {
|
||||
return Object.assign(err, { gitTimedOut: true });
|
||||
}
|
||||
|
||||
/**
|
||||
* Kill the whole child tree. POSIX uses the process group; Windows taskkill.
|
||||
*
|
||||
* Resolves when the kill OPERATION is finished, which on Windows means
|
||||
* taskkill has itself exited. That matters because taskkill walks the tree in
|
||||
* a separate process: the direct git child can close while taskkill is still
|
||||
* terminating its descendants, so a caller that settled on the child's close
|
||||
* alone could start deleting the workspace out from under processes that are
|
||||
* still running in it. On POSIX the group signal is delivered synchronously,
|
||||
* so there is nothing further to await.
|
||||
*
|
||||
* Never rejects: a kill that cannot be confirmed is reported through the
|
||||
* fallback warning, and the caller's own confirmation timeout bounds the wait.
|
||||
*/
|
||||
function killTree(child: ChildProcess | undefined): Promise<void> {
|
||||
if (!child?.pid) return Promise.resolve();
|
||||
if (process.platform === 'win32') {
|
||||
return new Promise<void>((resolve) => {
|
||||
const killer = spawn('taskkill', ['/pid', String(child.pid), '/T', '/F'], { windowsHide: true });
|
||||
// 'error' and 'close' can both fire for one spawn attempt; only the
|
||||
// first should trigger the fallback so a single kill never falls back
|
||||
// (or logs) twice.
|
||||
let finished = false;
|
||||
const finish = (why?: string) => {
|
||||
if (finished) return;
|
||||
finished = true;
|
||||
if (why) {
|
||||
console.warn(`[GitSource:transport] taskkill ${why} for pid ${child.pid}; falling back to child.kill() (tree-kill guarantee no longer holds: descendants of ${child.pid} may still be running)`);
|
||||
// This runs in an event callback, where a throw would be an
|
||||
// uncaught exception rather than a rejection of this promise.
|
||||
try {
|
||||
child.kill();
|
||||
} catch (e) {
|
||||
console.warn(`[GitSource:transport] fallback kill for pid ${child.pid} failed: ${e instanceof Error ? e.message : String(e)}`);
|
||||
}
|
||||
}
|
||||
resolve();
|
||||
};
|
||||
killer.on('error', (err) => finish(`failed to spawn (${err.message})`));
|
||||
killer.on('close', (code) => finish(code === 0 ? undefined : `exited ${code}`));
|
||||
});
|
||||
}
|
||||
try {
|
||||
// The child is spawned detached, so it leads its own process group.
|
||||
process.kill(-child.pid, 'SIGKILL');
|
||||
} catch {
|
||||
try {
|
||||
child.kill('SIGKILL');
|
||||
} catch (e) {
|
||||
console.warn(`[GitSource:transport] fallback kill for pid ${child.pid} failed: ${e instanceof Error ? e.message : String(e)}`);
|
||||
}
|
||||
}
|
||||
return Promise.resolve();
|
||||
}
|
||||
|
||||
/**
|
||||
* Await a kill that has already been issued, but never longer than
|
||||
* KILL_CONFIRM_TIMEOUT_MS. A platform kill helper that wedges must not turn
|
||||
* into a caller that hangs forever with nothing logged; past the bound we say
|
||||
* so and carry on, exactly as runGit's own confirmation timer does.
|
||||
*/
|
||||
async function awaitKillConfirmed(kill: Promise<void> | undefined, what: string): Promise<void> {
|
||||
if (!kill) return;
|
||||
let timer: NodeJS.Timeout | undefined;
|
||||
const bound = new Promise<void>((resolve) => {
|
||||
timer = setTimeout(() => {
|
||||
console.warn(`[GitSource:transport] ${what} not confirmed within ${KILL_CONFIRM_TIMEOUT_MS}ms; continuing cleanup while it may still be running`);
|
||||
resolve();
|
||||
}, KILL_CONFIRM_TIMEOUT_MS);
|
||||
});
|
||||
try {
|
||||
await Promise.race([kill, bound]);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
/** Bound on how long to wait for a confirmed close after a kill is issued, so a kill that never reports back cannot hang the caller forever. */
|
||||
const KILL_CONFIRM_TIMEOUT_MS = 5_000;
|
||||
|
||||
/**
|
||||
* Spawn git and collect output. On timeout, kills the entire child tree and
|
||||
* waits for its `close` event before rejecting: settling as soon as the kill
|
||||
* is merely issued (rather than confirmed) would let the caller start
|
||||
* cleaning up the workspace while the child tree, or the platform kill
|
||||
* helper (taskkill), is still running. Resolves with whatever exit code git
|
||||
* reported when it closes on its own.
|
||||
*/
|
||||
function runGit(args: string[], opts: RunOptions): Promise<RunResult> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn('git', args, {
|
||||
cwd: opts.cwd,
|
||||
env: opts.env,
|
||||
windowsHide: true,
|
||||
detached: process.platform !== 'win32',
|
||||
});
|
||||
let stdout = '';
|
||||
let stderr = '';
|
||||
let settled = false;
|
||||
let timedOut = false;
|
||||
let killConfirmTimer: NodeJS.Timeout | undefined;
|
||||
// Resolves once the kill operation itself is done (see killTree); a
|
||||
// timed-out run must not settle before BOTH this and the child's own
|
||||
// close event.
|
||||
let killFinished: Promise<void> | undefined;
|
||||
const append = (cur: string, chunk: Buffer) => (cur.length < STDERR_CAP ? cur + chunk.toString('utf8') : cur);
|
||||
|
||||
const timer = setTimeout(() => {
|
||||
if (settled) return;
|
||||
timedOut = true;
|
||||
killFinished = killTree(child);
|
||||
killConfirmTimer = setTimeout(() => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
console.error(`[GitSource:transport] kill not confirmed within ${KILL_CONFIRM_TIMEOUT_MS}ms for pid ${child.pid}; the process may still be running`);
|
||||
reject(asTimeoutError(new Error('git timed out (kill unconfirmed)')));
|
||||
}, KILL_CONFIRM_TIMEOUT_MS);
|
||||
}, opts.timeoutMs);
|
||||
|
||||
child.stdout?.on('data', (c: Buffer) => {
|
||||
stdout = append(stdout, c);
|
||||
});
|
||||
child.stderr?.on('data', (c: Buffer) => {
|
||||
stderr = append(stderr, c);
|
||||
});
|
||||
const finishSettle = (): boolean => {
|
||||
if (settled) return false;
|
||||
settled = true;
|
||||
clearTimeout(timer);
|
||||
clearTimeout(killConfirmTimer);
|
||||
return true;
|
||||
};
|
||||
|
||||
/**
|
||||
* Reject a timed-out run, but only once the kill operation has also
|
||||
* finished. The direct child can be gone while the platform kill
|
||||
* helper (taskkill) is still walking its descendants, and settling in
|
||||
* that window releases the caller to delete a workspace those
|
||||
* descendants are still using. Both settle paths below go through
|
||||
* here; the kill-confirmation timer above still bounds the wait.
|
||||
*/
|
||||
const rejectAfterKill = (err: Error): void => {
|
||||
void (killFinished ?? Promise.resolve()).then(() => {
|
||||
if (finishSettle()) reject(asTimeoutError(err));
|
||||
});
|
||||
};
|
||||
|
||||
child.on('error', (err) => {
|
||||
if (settled) return;
|
||||
// 'error' can also fire after the kill has been issued (e.g. the
|
||||
// process could not be killed); preserve the timeout flag so
|
||||
// callers still classify this as a timeout rather than a bare
|
||||
// exit failure, and wait for the kill exactly as 'close' does.
|
||||
if (timedOut) {
|
||||
rejectAfterKill(err);
|
||||
return;
|
||||
}
|
||||
if (finishSettle()) reject(err);
|
||||
});
|
||||
child.on('close', (code) => {
|
||||
if (settled) return;
|
||||
if (timedOut) {
|
||||
rejectAfterKill(new Error('git timed out'));
|
||||
return;
|
||||
}
|
||||
if (finishSettle()) resolve({ stdout, stderr, exitCode: code ?? -1 });
|
||||
});
|
||||
|
||||
opts.onSpawn?.(child);
|
||||
});
|
||||
}
|
||||
|
||||
// ─── Workspace layout ────────────────────────────────────────────────────────
|
||||
|
||||
interface WorkspaceLayout {
|
||||
/** Scratch dir for transport-owned files (credential helper script, combined CA bundle, isolated HOME). */
|
||||
metaDir: string;
|
||||
hooksDir: string;
|
||||
homeDir: string;
|
||||
}
|
||||
|
||||
async function prepareWorkspace(root: string): Promise<WorkspaceLayout> {
|
||||
const metaDir = path.join(root, '.meta');
|
||||
const hooksDir = path.join(metaDir, 'hooks');
|
||||
const homeDir = path.join(metaDir, 'home');
|
||||
await fs.mkdir(hooksDir, { recursive: true });
|
||||
await fs.mkdir(homeDir, { recursive: true });
|
||||
return { metaDir, hooksDir, homeDir };
|
||||
}
|
||||
|
||||
function buildEnv(homeDir: string, token?: string | null, helperPath?: string | null): NodeJS.ProcessEnv {
|
||||
const env: NodeJS.ProcessEnv = {
|
||||
...process.env,
|
||||
GIT_CONFIG_NOSYSTEM: '1',
|
||||
// GIT_CONFIG_NOSYSTEM does not block explicit config-file pointers or
|
||||
// XDG lookups; pin every channel git could read operator config from.
|
||||
GIT_CONFIG_GLOBAL: os.devNull,
|
||||
GIT_CONFIG_SYSTEM: os.devNull,
|
||||
XDG_CONFIG_HOME: '',
|
||||
GIT_CONFIG_COUNT: '0',
|
||||
GIT_TERMINAL_PROMPT: '0',
|
||||
GIT_ASKPASS: '',
|
||||
// An inherited trace flag would widen the log surface with packet
|
||||
// dumps that can carry URL material.
|
||||
GIT_TRACE: '',
|
||||
HOME: homeDir,
|
||||
};
|
||||
if (process.platform === 'win32') {
|
||||
env.USERPROFILE = homeDir;
|
||||
}
|
||||
if (token) {
|
||||
env[GIT_TOKEN_ENV_VAR] = token;
|
||||
}
|
||||
if (helperPath) {
|
||||
// The helper's location, kept out of the credential.helper config
|
||||
// value so no workspace path character can change how git's shell
|
||||
// parses it. See credentialHelper.ts.
|
||||
env[GIT_HELPER_PATH_ENV_VAR] = helperPath;
|
||||
}
|
||||
return env;
|
||||
}
|
||||
|
||||
/**
|
||||
* Windows-only: locate the CA bundle bundled with Git for Windows. Stripping
|
||||
* system gitconfig (GIT_CONFIG_NOSYSTEM) also strips the installer's
|
||||
* http.sslCAInfo pointer to this file, and unlike Linux there is no /etc/ssl
|
||||
* default for the OpenSSL backend to fall back on.
|
||||
*/
|
||||
async function detectWindowsCABundle(): Promise<string | null> {
|
||||
try {
|
||||
const execPath = await getGitExecPath();
|
||||
const installRoot = path.resolve(execPath, '..', '..'); // <install>/mingw64/libexec/git-core -> <install>/mingw64
|
||||
const candidates = [
|
||||
path.join(installRoot, 'etc', 'ssl', 'certs', 'ca-bundle.crt'),
|
||||
path.resolve(execPath, '..', '..', '..', 'usr', 'ssl', 'certs', 'ca-bundle.crt'),
|
||||
];
|
||||
for (const candidate of candidates) {
|
||||
if (existsSync(candidate)) return candidate.split(path.sep).join('/');
|
||||
}
|
||||
} catch {
|
||||
// Fall through: without a bundle the fetch fails with a clear TLS
|
||||
// classification instead of a silent trust downgrade.
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** First existing system CA bundle for OpenSSL-backed git on POSIX. */
|
||||
const POSIX_CA_BUNDLE_CANDIDATES = [
|
||||
'/etc/ssl/certs/ca-certificates.crt',
|
||||
'/etc/pki/tls/certs/ca-bundle.crt',
|
||||
];
|
||||
|
||||
/**
|
||||
* Build the CA-anchor configuration for one fetch.
|
||||
*
|
||||
* Mirrors Node's own NODE_EXTRA_CA_CERTS semantics (extra anchors ADDED to
|
||||
* the defaults, never replacing them) by writing a combined PEM bundle into
|
||||
* the fetch workspace's `.meta` dir:
|
||||
* - No NODE_EXTRA_CA_CERTS: production posture. POSIX passes nothing and
|
||||
* lets OpenSSL use system trust; Windows pins Git's own bundled bundle,
|
||||
* because stripping system gitconfig also strips the installer's pointer
|
||||
* to it.
|
||||
* - With NODE_EXTRA_CA_CERTS: defaults PLUS the extra CAs, so the dev/E2E
|
||||
* fixture server and public hosts validate in the same process state.
|
||||
*/
|
||||
async function resolveCaArgs(layout: WorkspaceLayout): Promise<string[]> {
|
||||
const extraPath = process.env.NODE_EXTRA_CA_CERTS;
|
||||
const hasExtra = Boolean(extraPath && existsSync(extraPath));
|
||||
const isWindows = process.platform === 'win32';
|
||||
|
||||
if (!hasExtra && !isWindows) {
|
||||
return [];
|
||||
}
|
||||
|
||||
if (isWindows && !hasExtra) {
|
||||
// Windows without an override: anchor to Git's bundled bundle directly.
|
||||
const bundle = await detectWindowsCABundle();
|
||||
return bundle ? ['-c', `http.sslCAInfo=${bundle}`] : [];
|
||||
}
|
||||
|
||||
let defaultPem = '';
|
||||
let winBundle: string | null = null;
|
||||
if (isWindows) {
|
||||
winBundle = await detectWindowsCABundle();
|
||||
if (winBundle) {
|
||||
try {
|
||||
defaultPem = await fs.readFile(winBundle.replace(/\//g, path.sep), 'utf8');
|
||||
} catch {
|
||||
console.warn(`[GitSource:transport] could not read system CA bundle at ${winBundle}; combined anchors will contain only NODE_EXTRA_CA_CERTS entries.`);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
for (const candidate of POSIX_CA_BUNDLE_CANDIDATES) {
|
||||
if (!existsSync(candidate)) continue;
|
||||
try {
|
||||
defaultPem = await fs.readFile(candidate, 'utf8');
|
||||
break;
|
||||
} catch {
|
||||
// Try the next candidate.
|
||||
}
|
||||
}
|
||||
if (!defaultPem) {
|
||||
console.warn('[GitSource:transport] no readable system CA bundle found; combined anchors will contain only NODE_EXTRA_CA_CERTS entries.');
|
||||
}
|
||||
}
|
||||
|
||||
let extraPem = '';
|
||||
try {
|
||||
extraPem = await fs.readFile(extraPath as string, 'utf8');
|
||||
} catch {
|
||||
console.warn('[GitSource:transport] could not read the file configured via NODE_EXTRA_CA_CERTS; ignoring custom anchors.');
|
||||
// Windows still has working defaults; fall back to them instead of
|
||||
// dropping every anchor.
|
||||
return isWindows && winBundle ? ['-c', `http.sslCAInfo=${winBundle}`] : [];
|
||||
}
|
||||
const combinedPath = path.join(layout.metaDir, 'combined-ca.pem');
|
||||
await fs.writeFile(combinedPath, `${defaultPem}\n${extraPem}`, { mode: 0o600 });
|
||||
return ['-c', `http.sslCAInfo=${combinedPath.split(path.sep).join('/')}`];
|
||||
}
|
||||
|
||||
/**
|
||||
* Config shared by every invocation. With no helper, credential.helper is
|
||||
* explicitly cleared so nothing from the environment can answer prompts.
|
||||
*/
|
||||
async function commonArgs(layout: WorkspaceLayout, helperPath: string | null): Promise<string[]> {
|
||||
const args = [
|
||||
'-c', 'protocol.allow=never',
|
||||
'-c', 'protocol.https.allow=always',
|
||||
'-c', `core.hooksPath=${layout.hooksDir.split(path.sep).join('/')}`,
|
||||
];
|
||||
if (process.platform === 'win32') {
|
||||
// With every config channel neutralized above, git falls back to its
|
||||
// build-default TLS backend, which on Git for Windows can be
|
||||
// schannel. Schannel ignores http.sslCAInfo (breaking the dev/E2E CA
|
||||
// bridge) and trusts per-Windows-cert-store state, so pin the
|
||||
// OpenSSL backend that ships with Git for Windows. Production Alpine
|
||||
// git is OpenSSL-backed and unaffected by this flag's absence.
|
||||
args.push('-c', 'http.sslBackend=openssl');
|
||||
}
|
||||
args.push(...await resolveCaArgs(layout));
|
||||
if (helperPath !== null) {
|
||||
// A fixed value: the helper's path reaches git through the child env
|
||||
// instead of being interpolated here, so a workspace path containing
|
||||
// a space (or a quote, `$`, `;`, ...) cannot change how git's shell
|
||||
// parses it. See credentialHelper.ts for the parsing rule.
|
||||
args.push('-c', `credential.helper=${CREDENTIAL_HELPER_CONFIG_VALUE}`);
|
||||
} else {
|
||||
args.push('-c', 'credential.helper=');
|
||||
}
|
||||
return args;
|
||||
}
|
||||
|
||||
/**
|
||||
* Everything one invocation needs: the workspace layout, the child env, and
|
||||
* the shared config argv.
|
||||
*
|
||||
* The credential handoff is assembled in exactly this one place because its
|
||||
* three parts have to agree. If the config named the helper variable but the
|
||||
* env did not export it, git would find nothing to run, fall back to an
|
||||
* anonymous fetch, and a private repo's 401 would then classify as
|
||||
* REPO_NOT_FOUND instead of AUTH_FAILED. That is a silent downgrade, so the
|
||||
* config arg is keyed off the helper actually having been written rather than
|
||||
* off the token being present.
|
||||
*/
|
||||
async function prepareInvocation(
|
||||
workspaceRoot: string,
|
||||
token?: string | null,
|
||||
): Promise<{ layout: WorkspaceLayout; env: NodeJS.ProcessEnv; baseArgs: string[] }> {
|
||||
const layout = await prepareWorkspace(workspaceRoot);
|
||||
const helperPath = token ? await writeCredentialHelper(layout.metaDir) : null;
|
||||
const env = buildEnv(layout.homeDir, token, helperPath);
|
||||
// The same helperPath drives the env export and the config arg, so the two
|
||||
// cannot describe different worlds.
|
||||
const baseArgs = await commonArgs(layout, helperPath);
|
||||
return { layout, env, baseArgs };
|
||||
}
|
||||
|
||||
// ─── Input validation ────────────────────────────────────────────────────────
|
||||
|
||||
function invalidUrl(host: string, hasToken: boolean): TransportFailure {
|
||||
return { transportFailure: true as const, reason: 'invalid-url', host, hasToken };
|
||||
}
|
||||
|
||||
function assertValidRepoUrl(repoUrl: string, hasToken: boolean): URL {
|
||||
let url: URL;
|
||||
try {
|
||||
url = new URL(repoUrl);
|
||||
} catch {
|
||||
throw invalidUrl('unknown', hasToken);
|
||||
}
|
||||
if (url.protocol !== 'https:' || !url.hostname || url.username || url.password) {
|
||||
throw invalidUrl(url.host || 'unknown', hasToken);
|
||||
}
|
||||
return url;
|
||||
}
|
||||
|
||||
/**
|
||||
* Ceiling on a ref name. Git itself imposes no branch-length limit worth
|
||||
* matching (`check-ref-format --branch` accepts names into the thousands, up
|
||||
* to the filesystem's own path limits), so this is Sencho's bound, not git's,
|
||||
* and the API and the transport have to agree on it: a name the route accepts
|
||||
* and stores must not be rejected later by the transport that fetches it.
|
||||
* `routes/gitSources.ts` imports this constant for exactly that reason.
|
||||
*/
|
||||
export const REF_MAX_LEN = 256;
|
||||
// ASCII control characters, space, and the characters git's own
|
||||
// check-ref-format forbids anywhere in a ref (~^:?*[\). Everything else,
|
||||
// including non-ASCII scripts, is a legitimate branch-name character.
|
||||
const REF_DISALLOWED_CHARS = /[\x00-\x20\x7f~^:?*[\\]/;
|
||||
|
||||
/**
|
||||
* Validates a ref name against the rules `git check-ref-format --branch`
|
||||
* applies to a branch: no control characters, space, or `~^:?*[\`; no `..`
|
||||
* or `@{`; no path component starting with `.` or ending in `.lock`; no
|
||||
* leading `-` (git's own `--branch` mode already refuses this, since a
|
||||
* leading dash makes the name ambiguous with a flag on argv, which is
|
||||
* exactly the injection risk this validator exists to close). The one rule
|
||||
* that is ours rather than git's is REF_MAX_LEN, shared with the route so the
|
||||
* two cannot disagree.
|
||||
*/
|
||||
function assertValidRef(ref: string, host: string, hasToken: boolean): void {
|
||||
const segments = ref.split('/');
|
||||
const valid = ref.length > 0
|
||||
&& ref.length <= REF_MAX_LEN
|
||||
&& !ref.startsWith('-')
|
||||
&& !REF_DISALLOWED_CHARS.test(ref)
|
||||
&& !ref.includes('..')
|
||||
&& !ref.includes('@{')
|
||||
&& !ref.endsWith('.')
|
||||
&& segments.every((seg) => seg.length > 0 && !seg.startsWith('.') && !seg.endsWith('.lock'));
|
||||
if (!valid) {
|
||||
throw { transportFailure: true as const, reason: 'invalid-ref', host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
}
|
||||
|
||||
// ─── Size watchdog ───────────────────────────────────────────────────────────
|
||||
|
||||
async function treeSize(root: string): Promise<number> {
|
||||
let total = 0;
|
||||
const stack: string[] = [root];
|
||||
while (stack.length) {
|
||||
const dir = stack.pop();
|
||||
if (!dir) break;
|
||||
const entries = await fs.readdir(dir, { withFileTypes: true });
|
||||
for (const entry of entries) {
|
||||
const p = path.join(dir, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
stack.push(p);
|
||||
} else if (entry.isFile()) {
|
||||
total += (await fs.stat(p)).size;
|
||||
}
|
||||
// Symlinks are neither followed nor counted; clones made here do
|
||||
// not create them and counting targets could inflate the sum.
|
||||
}
|
||||
}
|
||||
return total;
|
||||
}
|
||||
|
||||
interface Watchdog {
|
||||
stop(): void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Polls the workspace's on-disk size and fires `onBreach` once it exceeds
|
||||
* `maxBytes`. Exported for tests.
|
||||
*/
|
||||
export function startSizeWatchdog(
|
||||
root: string,
|
||||
maxBytes: number,
|
||||
onBreach: () => void,
|
||||
): Watchdog {
|
||||
let stopped = false;
|
||||
let busy = false;
|
||||
let readFailures = 0;
|
||||
const timer = setInterval(() => {
|
||||
if (stopped || busy) return;
|
||||
busy = true;
|
||||
void treeSize(root)
|
||||
.then((size) => {
|
||||
readFailures = 0;
|
||||
if (!stopped && size > maxBytes) {
|
||||
stopped = true;
|
||||
clearInterval(timer);
|
||||
onBreach();
|
||||
}
|
||||
})
|
||||
.catch(() => {
|
||||
// Benign right after teardown (ENOENT mid-walk), but a
|
||||
// persistent inability to measure the workspace silently
|
||||
// disables the documented cap; say so once per streak.
|
||||
readFailures += 1;
|
||||
if (!stopped && readFailures === 3) {
|
||||
console.warn('[GitSource:transport] could not stat clone workspace; GITSOURCE_MAX_CLONE_BYTES enforcement is degraded for this fetch.');
|
||||
}
|
||||
})
|
||||
.finally(() => {
|
||||
busy = false;
|
||||
});
|
||||
}, WATCHDOG_INTERVAL_MS);
|
||||
return {
|
||||
stop(): void {
|
||||
stopped = true;
|
||||
clearInterval(timer);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// ─── Transport implementation ────────────────────────────────────────────────
|
||||
|
||||
async function ensureBinaryReady(hasToken: boolean): Promise<void> {
|
||||
try {
|
||||
await ensureGitBinary();
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e);
|
||||
const stale = /too old/.test(message);
|
||||
throw { transportFailure: true as const, reason: stale ? 'git-old' : 'git-missing', stderr: message, host: 'unknown', hasToken } satisfies TransportFailure;
|
||||
}
|
||||
}
|
||||
|
||||
function parseLsRemoteLine(line: string, fullRef: string): string | null {
|
||||
const tabIndex = line.indexOf('\t');
|
||||
if (tabIndex === -1) return null;
|
||||
if (line.slice(tabIndex + 1).trim() !== fullRef) return null;
|
||||
const sha = line.slice(0, tabIndex).trim();
|
||||
return SHA_PATTERN.test(sha) ? sha.toLowerCase() : null;
|
||||
}
|
||||
|
||||
async function lsRemoteHead(
|
||||
url: URL,
|
||||
ref: string,
|
||||
env: NodeJS.ProcessEnv,
|
||||
baseArgs: string[],
|
||||
timeoutMs: number,
|
||||
hasToken: boolean,
|
||||
): Promise<string> {
|
||||
let res: RunResult;
|
||||
try {
|
||||
res = await runGit(
|
||||
[...baseArgs, 'ls-remote', '--heads', url.href, `refs/heads/${ref}`],
|
||||
{ env, timeoutMs: Math.min(timeoutMs, LS_REMOTE_MAX_MS) },
|
||||
);
|
||||
} catch (e) {
|
||||
// A resolution-phase timeout must classify like any other network
|
||||
// timeout, not leak the internal flagged error to callers.
|
||||
if (isTimeoutError(e)) {
|
||||
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
if (res.exitCode !== 0) {
|
||||
throw { transportFailure: true as const, reason: 'exit', stderr: res.stderr, exitCode: res.exitCode, argv: baseArgs, host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
const fullRef = `refs/heads/${ref}`;
|
||||
for (const line of res.stdout.split(/\r?\n/)) {
|
||||
const sha = parseLsRemoteLine(line, fullRef);
|
||||
if (sha) return sha;
|
||||
}
|
||||
throw { transportFailure: true as const, reason: 'ref-not-found', host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
|
||||
export const nativeGitTransport: GitTransport = {
|
||||
async resolveRef(req: ResolveRequest): Promise<{ commitSha: string }> {
|
||||
const hasToken = Boolean(req.token);
|
||||
await ensureBinaryReady(hasToken);
|
||||
const url = assertValidRepoUrl(req.repoUrl, hasToken);
|
||||
assertValidRef(req.ref, url.host, hasToken);
|
||||
|
||||
const { env, baseArgs } = await prepareInvocation(req.workspaceRoot, req.token);
|
||||
const commitSha = await lsRemoteHead(
|
||||
url, req.ref, env, baseArgs,
|
||||
req.timeoutMs ?? DEFAULT_TIMEOUT_MS, hasToken,
|
||||
);
|
||||
return { commitSha };
|
||||
},
|
||||
|
||||
async fetchAtCommit(req: FetchRequest): Promise<FetchResult> {
|
||||
const hasToken = Boolean(req.token);
|
||||
await ensureBinaryReady(hasToken);
|
||||
const url = assertValidRepoUrl(req.repoUrl, hasToken);
|
||||
assertValidRef(req.ref, url.host, hasToken);
|
||||
|
||||
const { layout, env, baseArgs } = await prepareInvocation(req.workspaceRoot, req.token);
|
||||
const checkout = path.join(req.workspaceRoot, 'repo');
|
||||
const timeoutMs = req.timeoutMs ?? DEFAULT_TIMEOUT_MS;
|
||||
|
||||
// Shared between the watchdog breach flag and the active child handle:
|
||||
// whichever fires first tears the clone down; the other becomes a no-op.
|
||||
let sizeExceeded = false;
|
||||
let activeChild: ChildProcess | undefined;
|
||||
// A breach kill is subject to the same ordering hazard as a timeout
|
||||
// kill: the caller deletes this workspace as soon as we return, so the
|
||||
// kill has to be finished first, not merely issued.
|
||||
let breachKill: Promise<void> | undefined;
|
||||
const watchdog = startSizeWatchdog(req.workspaceRoot, req.maxBytes, () => {
|
||||
sizeExceeded = true;
|
||||
breachKill = killTree(activeChild);
|
||||
});
|
||||
|
||||
try {
|
||||
let cloneResult: RunResult;
|
||||
try {
|
||||
cloneResult = await runGit(
|
||||
[
|
||||
...baseArgs, 'clone',
|
||||
'--depth=1', '--single-branch', '--no-tags', '--no-recurse-submodules',
|
||||
'--branch', req.ref, url.href, checkout,
|
||||
],
|
||||
{ cwd: layout.homeDir, env, timeoutMs, onSpawn: (child) => { activeChild = child; } },
|
||||
);
|
||||
} catch (e) {
|
||||
// A size breach wins over the timeout wording: both kills are
|
||||
// ours, but the operator guidance differs.
|
||||
if (sizeExceeded) {
|
||||
throw { transportFailure: true as const, reason: 'size', maxBytes: req.maxBytes, host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
if (isTimeoutError(e)) {
|
||||
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
throw { transportFailure: true as const, reason: 'exit', stderr: e instanceof Error ? e.message : String(e), argv: [...baseArgs, 'clone'], host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
|
||||
// A watchdog-triggered SIGKILL settles runGit's promise via the
|
||||
// child's normal 'close' event (code null -> exitCode -1), not a
|
||||
// rejection, so this branch is the common path for an in-flight
|
||||
// breach and must check sizeExceeded before the generic mapping.
|
||||
if (sizeExceeded) {
|
||||
throw { transportFailure: true as const, reason: 'size', maxBytes: req.maxBytes, host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
|
||||
// runGit resolves on any exit code: a failed clone (auth, missing
|
||||
// repo, TLS) must classify by its real stderr here rather than
|
||||
// fall through to rev-parse and surface as a generic GIT_ERROR.
|
||||
if (cloneResult.exitCode !== 0) {
|
||||
throw {
|
||||
transportFailure: true as const,
|
||||
reason: 'exit',
|
||||
stderr: cloneResult.stderr,
|
||||
exitCode: cloneResult.exitCode,
|
||||
argv: [...baseArgs, 'clone'],
|
||||
host: url.host,
|
||||
hasToken,
|
||||
} satisfies TransportFailure;
|
||||
}
|
||||
|
||||
let actual: string;
|
||||
try {
|
||||
const head = await runGit([...baseArgs, 'rev-parse', 'HEAD'], {
|
||||
cwd: checkout,
|
||||
env,
|
||||
timeoutMs: Math.min(timeoutMs, LS_REMOTE_MAX_MS),
|
||||
});
|
||||
actual = head.stdout.trim().toLowerCase();
|
||||
if (!SHA_PATTERN.test(actual)) {
|
||||
throw { transportFailure: true as const, reason: 'exit', stderr: `unexpected rev-parse output: ${head.stdout}`, exitCode: head.exitCode, host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
} catch (e) {
|
||||
if (isTransportFailure(e)) throw e;
|
||||
if (isTimeoutError(e)) {
|
||||
throw { transportFailure: true as const, reason: 'timeout', host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
throw { transportFailure: true as const, reason: 'exit', stderr: e instanceof Error ? e.message : String(e), host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
|
||||
if (actual !== req.commitSha.toLowerCase()) {
|
||||
// The branch tip moved between resolution and fetch. Refuse
|
||||
// rather than materialize content nobody reviewed.
|
||||
throw { transportFailure: true as const, reason: 'tip-changed', host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
|
||||
// Deterministic final measure: a breach landing between the last
|
||||
// watchdog tick and successful verification must not slip through
|
||||
// as an over-cap success. A read failure here (permissions,
|
||||
// workspace removed mid-walk) must fail closed rather than treat
|
||||
// an unmeasurable workspace as within budget.
|
||||
const finalSize = await treeSize(req.workspaceRoot).catch((e: unknown) => {
|
||||
console.warn(`[GitSource:transport] final size measurement failed for ${req.workspaceRoot}, failing closed: ${e instanceof Error ? e.message : String(e)}`);
|
||||
return -1;
|
||||
});
|
||||
if (sizeExceeded || finalSize < 0 || finalSize > req.maxBytes) {
|
||||
throw { transportFailure: true as const, reason: 'size', maxBytes: req.maxBytes, host: url.host, hasToken } satisfies TransportFailure;
|
||||
}
|
||||
|
||||
return { commitSha: actual, dir: checkout };
|
||||
} finally {
|
||||
watchdog.stop();
|
||||
await awaitKillConfirmed(breachKill, `size-breach kill for ${url.host}`);
|
||||
}
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,57 @@
|
||||
/**
|
||||
* Adapter contract between GitSourceService and the native git transport.
|
||||
*
|
||||
* Resolution is separated from fetch on purpose: every pull resolves the
|
||||
* configured ref to an immutable commit BEFORE any content is downloaded,
|
||||
* and the fetch verifies it landed on exactly that commit. That makes
|
||||
* immutable resolution structural rather than a convention callers have to
|
||||
* remember. The ref field carries branch names today; widening it to tags and
|
||||
* pinned SHAs later does not change either method's shape.
|
||||
*/
|
||||
|
||||
export interface ResolveRequest {
|
||||
repoUrl: string;
|
||||
/** Branch names today; tags and pinned SHAs may widen this later. */
|
||||
ref: string;
|
||||
token?: string | null;
|
||||
/**
|
||||
* Total fetch budget in milliseconds. Note: the resolution round trip
|
||||
* (ls-remote) is internally capped at 10s regardless of this value, so
|
||||
* the worst-case wall clock is roughly clamp(resolve) + full clone, plus
|
||||
* up to a further 5s per timed-out invocation while the transport waits
|
||||
* for confirmed child-process termination before giving up.
|
||||
*/
|
||||
timeoutMs?: number;
|
||||
/**
|
||||
* Caller-owned temp workspace root. Resolution authenticates too (a
|
||||
* private repo hides its refs from anonymous ls-remote), so it needs a
|
||||
* place for the credential helper.
|
||||
*/
|
||||
workspaceRoot: string;
|
||||
}
|
||||
|
||||
export interface FetchRequest extends ResolveRequest {
|
||||
/**
|
||||
* Commit produced by resolveRef; the fetched checkout is verified to be
|
||||
* exactly this commit before it can be used. `ref` locates what to fetch,
|
||||
* `commitSha` pins what may be trusted.
|
||||
*/
|
||||
commitSha: string;
|
||||
/** Ceiling for the on-disk clone; enforced by the size watchdog. */
|
||||
maxBytes: number;
|
||||
}
|
||||
|
||||
export interface FetchResult {
|
||||
commitSha: string;
|
||||
/** Checked-out working tree, ready for read-only inspection. */
|
||||
dir: string;
|
||||
}
|
||||
|
||||
export interface ResolveResult {
|
||||
commitSha: string;
|
||||
}
|
||||
|
||||
export interface GitTransport {
|
||||
resolveRef(req: ResolveRequest): Promise<ResolveResult>;
|
||||
fetchAtCommit(req: FetchRequest): Promise<FetchResult>;
|
||||
}
|
||||
@@ -246,7 +246,7 @@ Pulls, applies, and create-from-git operations on the same stack are serialized
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Repository too large">
|
||||
A clone is capped on how much it downloads (100 MB by default), and individual compose/env files are capped on read. A compose repository is normally tiny, so hitting either usually means the tracked branch carries large binaries. Point the source at a repository or branch that holds just your compose and `.env` files. If a large repository is unavoidable, an operator can raise the download ceiling with the `GITSOURCE_MAX_CLONE_BYTES` environment variable.
|
||||
A clone is capped on the on-disk size of its temporary workspace (100 MB by default), and individual compose/env files are capped on read. A compose repository is normally tiny, so hitting either usually means the tracked branch carries large binaries. Point the source at a repository or branch that holds just your compose and `.env` files. If a large repository is unavoidable, an operator can raise the workspace ceiling with the `GITSOURCE_MAX_CLONE_BYTES` environment variable.
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Pending commit has changed since this pull was fetched">
|
||||
@@ -284,7 +284,7 @@ Pulls, applies, and create-from-git operations on the same stack are serialized
|
||||
- **No Git LFS.** Compose and env files stored via LFS are rejected. Commit plain files instead.
|
||||
- **No submodules.** Submodule contents are not fetched. Inputs and build contexts that reference submodule contents are refused with an actionable message; a warning is shown when `.gitmodules` is present.
|
||||
- **Branch-tracking only.** Sources follow the head of a branch. Specific commit SHAs and tags are not pinnable. Each pull resolves and pins the exact commit SHA, so apply always materializes the reviewed revision.
|
||||
- **Clone size cap.** A clone is bounded on how much it downloads (and each compose/env file is capped on read), so very large repositories are rejected. Operators can adjust the download ceiling with `GITSOURCE_MAX_CLONE_BYTES`.
|
||||
- **Clone size cap.** A clone is bounded on the on-disk size of its temporary workspace (and each compose/env file is capped on read), so very large repositories are rejected. Operators can adjust the workspace ceiling with `GITSOURCE_MAX_CLONE_BYTES`.
|
||||
- **Complete project materialization.** Every repository-local input the project needs is materialized: the ordered compose files, implicit `compose.override.*` files, recursive `include:` and `extends.file` dependencies, service env files, file-backed configs and secrets, label files, and build contexts with `.dockerignore` semantics. The materialized set is recorded in a versioned managed-project manifest, and each pull stages a candidate that is validated with the exact deployment invocation before anything on disk changes. If apply is interrupted, Sencho completes the accepted generation or restores the previous generation. If files were edited during the interruption and no longer match either generation, Sencho preserves them and requires manual recovery instead of overwriting them.
|
||||
- **Unsupported inputs are refused, not guessed.** Inputs that cannot be safely reproduced fail the pull with an actionable message: URL includes, Git LFS pointers, submodule contents, symbolic links, build contexts that exceed the size bounds, and include or extends declarations that point outside the repository or use dynamic `\${VAR}` paths (their contents cannot be enumerated). Nothing is applied until the declaration is fixed. Absolute host paths, host bind mounts, external resources, and dynamic `\${VAR}` data paths are never claimed as covered: they resolve at deploy time from the environment or the node, and the manifest records them as unmanaged.
|
||||
- **Materialization bounds.** The materialized project is bounded by file count, total bytes, per-file size, path depth, and build-context size, each adjustable with a `GITSOURCE_*` variable (see configuration). Crossing a bound refuses the pull with the counts so far rather than producing a partial project.
|
||||
|
||||
@@ -48,7 +48,7 @@ These tune optional subsystems. Most deployments never set them; the defaults ar
|
||||
| `SENCHO_MESH_SUBNET` | *(auto)* | CIDR for this node's `sencho_mesh` network. When unset, Sencho picks the first free `/24` from its candidate list or adopts an existing mesh subnet. Set one only to avoid an overlap with another network on the host. See [Sencho Mesh](/features/sencho-mesh). |
|
||||
| `SENCHO_MESH_RECONCILE_INTERVAL_MS` | `60000` | How often the central instance re-checks proxy-mode mesh tunnels to detect a peer that rebooted. Lower it for faster peer-reboot detection at the cost of more frequent checks. See [Sencho Mesh](/features/sencho-mesh). |
|
||||
| `SENCHO_MESH_PROXY_TUNNEL_IDLE_MS` | `0` | Idle timeout before a proxy-mode mesh tunnel tears down and reopens on demand. `0` keeps the tunnel open for the life of the connection. See [Sencho Mesh](/features/sencho-mesh). |
|
||||
| `GITSOURCE_MAX_CLONE_BYTES` | `104857600` | Maximum bytes a single [Git Source](/features/git-sources) clone may download before it is aborted (100 MB). A shallow Compose clone is tiny; raise it only if you track Compose files in a legitimately large repository. |
|
||||
| `GITSOURCE_MAX_CLONE_BYTES` | `104857600` | Maximum on-disk size of the temporary workspace for a single [Git Source](/features/git-sources) clone before it is aborted (100 MB). A shallow Compose clone is tiny; raise it only if you track Compose files in a legitimately large repository. |
|
||||
| `GITSOURCE_MAX_MATERIALIZED_FILES` | `10000` | Maximum number of files the complete-project materializer may write for one Git-managed stack. Crossing it refuses the pull with the count so far. |
|
||||
| `GITSOURCE_MAX_MATERIALIZED_BYTES` | `536870912` | Maximum total bytes the complete-project materializer may write for one Git-managed stack (512 MB). Crossing it refuses the pull with the count so far. |
|
||||
| `GITSOURCE_MAX_BUILD_CONTEXT_BYTES` | `268435456` | Maximum size of a single materialized build context after `.dockerignore` filtering (256 MB). A context that exceeds it, including a repository-root context, is refused. |
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
*
|
||||
* Builds fixture repositories with the system git binary and serves them over
|
||||
* HTTPS, so the full clone -> pull -> apply pipeline runs without network
|
||||
* egress. Implements the two smart-HTTP endpoints isomorphic-git needs
|
||||
* egress. Implements the two smart-HTTP endpoints the git CLI needs
|
||||
* (GET info/refs advertise + POST upload-pack) directly; git-http-backend's
|
||||
* stream internals break on modern Node.
|
||||
*
|
||||
|
||||
Reference in New Issue
Block a user