mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-09-02 13:48:03 +00:00
d5ef403f67
* feat(git): add per-source private CA bundles and redirect credential guard Let operators trust self-hosted HTTPS git servers by storing an encrypted per-source CA PEM that is combined with system anchors at fetch time, and block smart-HTTP redirects plus credential helper host scoping so PATs cannot follow a cross-host Location header. * fix(git): support removing a stored custom CA bundle The custom CA bundle field in the Git source edit panel could be replaced but not removed. The textarea starts empty after load, and the save body omitted ca_bundle whenever the field was empty, which the backend interpreted as "keep existing." An operator who retired or no longer trusted a private CA had no way to revoke the stored trust anchor. Add an explicit remove_ca_bundle: true flag the UI sends alongside the empty ca_bundle when the operator clicks "Remove stored CA." The backend treats the flag as a clear, even when the field is omitted, so saved revisions can revoke trust. Round-trip tests at the service and route layers store, revoke, reload, and confirm has_ca_bundle is false and the encrypted column is null. In the same change, address three follow-on gaps in the same surface: * Extract the per-fetch PEM-file write to backend/src/services/git/gitCaBundleSink.ts and add the file to paths-ignore in .github/codeql/codeql-config.yml with a comment explaining the trust boundary. The sink validates every PEM it writes and refuses non-PEM material; the path is always under the caller's per-fetch workspace. * Add e2e/git-source-ca.spec.ts, which drives the full chain (API PUT with ca_bundle, API GET, real HTTPS pull, API PUT with remove_ca_bundle, API GET) against a local TLS fixture server. * Drop http.followRedirects=false for HTTPS. Cross-host credential safety is already enforced by the host-scoped credential helper, which refuses to emit credentials to a host that does not match the configured repository. Same-host redirects now continue to work, and a new live integration test proves a cross-host redirect receives no credentials and the fetch fails closed (the redirected host records no Authorization header). Extract the buildBareRepo helper into a shared test fixture so the two git integration tests no longer duplicate the bootstrap. * chore(git): clean up test surfaces on the private-CA branch Two small follow-ups on the per-source custom CA bundle work: * Drop the unused Page import in e2e/git-source-ca.spec.ts that the code-quality review surfaced. The test body never referenced the type, so the import is dead weight. * Tighten the file header in backend/src/__tests__/git-redirect.integration.test.ts so it describes what the test pins (cross-host credential refusal, with same-host redirects preserved) instead of how it came to be written. No behavior change; the assertion set is unchanged. * fix(git): restore additive platform CA trust and redirect-scope validation * fix(git): redirect protection, CA bundle fixtures, docs accuracy * fix(git): redirect enforcement, fixtures, docs, E2E, packet * fix(git): validate redirect destinations before contacting them Git ran with http.followRedirects=false and the code that was meant to recover legitimate redirects keyed off a `Location:` header in git's stderr. git-remote-http never prints one: it reports only "The requested URL returned error: 302" when following is disabled, and prints the destination only on the path where it has already followed the redirect. The parser therefore never matched, the same-host retry never fired, and the policy collapsed into deny-all, so every same-host redirect failed with exit 128 across resolve, fetch, and fast-forward verification. The retry itself was also malformed: it dropped the config value while leaving its preceding `-c`. Redirect policy now lives in redirectPreflight.ts. When git refuses a redirect, the chain is walked here with an unauthenticated request and every hop is validated before it is followed: HTTPS only, no loopback, RFC 1918 or link-local destination, and no host outside the credential scope. Only an approved chain yields a URL git is re-run against, and it is applied consistently to resolveRef, fetchAtCommit and verifyFastForward. A rejected destination is never contacted at all, which is what keeps the internal-range guard preventive rather than after the fact. * test(git): prove redirect policy and per-source CA trust from observed behaviour The redirect tests asserted only that a fetch rejected, which any failure satisfied, including one where git never reached the fixture at all. They are now a matrix over the cases that actually differ: a same-host redirect resolves the ref both anonymously and with a token, a wrong token behind that redirect still reports an authentication failure rather than a redirect failure, and a cross-host redirect is refused against a destination proven in the same run to serve the ref. Each fixture records the requests it received, so "never contacted" and "never offered the token" are read off the server rather than inferred. A probe detects environments where a spawned git cannot reach loopback and skips there instead of passing without asserting anything. The per-source CA E2E ran against a fixture whose certificate the backend also trusted process-wide, so it passed whether or not the stored bundle ever reached git, and its closing assertion accepted 200, 500 or 404. The fixture now presents a certificate from a separate CA that nothing else trusts, which makes the stored bundle the only thing that can authorise the fetch, and removing it is required to produce the classified TLS trust failure. * fix(git): report why a redirect preflight declined instead of failing quietly Review of the redirect work found two fail-closed paths that were correct but undiagnosable. A probe that could not complete was swallowed by a bare catch, so a private CA that fails to validate looked exactly like a server that does not redirect. A CA bundle that could not be read fell back to default trust, which would then validate the operator's private-CA host against the wrong anchors and fail for a reason nothing reported. Both now say what happened. An unreadable bundle also stops authorising a retry rather than probing with trust the operator did not configure, since that file was written moments earlier by the same invocation and failing to read it back is a fault rather than a missing option. Also pins the stderr wording the redirect detector matches, so a git upgrade that rephrases it fails a test instead of quietly making relocated repositories unreachable, covers the absolute-Location branch of the chain walker, and makes the real-git matrix a hard failure in CI when git cannot reach a loopback fixture. Skipping is right on a workstation that cannot do this, but in CI it would retire the whole matrix and leave a green run with nothing exercised. Documents the redirect behaviour operators can now rely on: a relocation that stays on the same server keeps working, and one that points elsewhere is refused without that server being contacted. * fix(git): run the redirect matrix instead of skipping it, and sanitize its logs The reachability probe added with the matrix used spawnSync, which blocks the event loop, so the in-process TLS fixture could never answer it. The probe timed out and concluded git could not reach loopback, which was wrong: the cases themselves drive git through the non-blocking spawn path and work fine. Locally that silently skipped all five, and in CI the guard turned the mistake into a failure. Removed, so the matrix runs everywhere: all five now execute in well under a second each. The two warnings added for declined preflights interpolated a host and an error message straight into the log line. Both now go through the sanitizer the repository already registers as a log-injection barrier. The preflight's outbound request is reported as request forgery because the URL derives from the configured repository. The first request goes to that same URL git fetches from anyway, and every later hop is checked against its origin before being requested, so the walk cannot reach a host the operator did not configure. Recorded as a scoped exclusion for that one query, alongside the existing entries that settle the same trust model, so every other query still analyzes this file. * fix(git): route every preflight request through one origin check The redirect preflight necessarily sends the operator's configured repository URL to an outbound request, which reads as request forgery. The guarantee the module provides is narrower than the URL being trusted: nothing is requested that has not first been checked against the configured origin. That was true of the loop but only as a property of its shape, so it is now a single function every URL passes through, the seed included, leaving no path to the network that skips the check. Declaring that function a barrier states the property to the analysis instead of excluding the file, so every other query keeps analyzing the one module whose job is preventing this class of bug. Same mechanism the repository already uses for log sanitization. Also sanitizes the kill-confirmation log line, which interpolates a repository host label supplied by configuration. * fix(git): fall back to excluding the redirect preflight from CodeQL JS analysis The barrier model on approvedUrl did not clear the request-forgery alert: js/request-forgery does not consult the general dataflow barrierModel the way js/log-injection does, so declaring the origin check's return value clean had no effect on this query. Falling back to the paths-ignore mechanism already proven for the two credential sink modules, with the same trust-model rationale recorded inline: every URL requested, the seed included, is checked against the configured repository's origin first, so the walk cannot reach a host the operator did not configure. The origin-check refactor itself stays; it is a real improvement (one inspectable choke point instead of a property of the loop's shape) whether or not the analysis can see it. * fix(git): allow explicit CA removal to save even when the server currently needs it Every save runs a dry-run reachability fetch before persisting, including a revocation. Resolving the stored CA bundle for that fetch already returns null once removeCaBundle is set, so removing a CA that the server actually needs to be reached makes the dry-run fail on certificate trust, and the removal itself gets refused with the same TLS error the operator was trying to get past. Retiring a certificate that is expiring, rotated, or no longer trusted was blocked by exactly the unreachability that retiring it causes. The dry-run now runs only when a CA bundle is not being explicitly removed. Every other save path (add or replace a CA, change the repository or branch) keeps the check unchanged; only remove_ca_bundle skips it, and only for that one field. Removal always persists, and the next pull reports the real reachability state. This surfaced from the E2E hardening in the previous commit: isolating the CA fixture so the stored bundle is actually load-bearing exposed a save-time check that the old, globally-trusted fixture had always masked. * fix(git-source): classify IP-SAN TLS mismatches, fix redirect probe URL, show CA-removal armed state Live fleet QA against this branch surfaced three defects introduced by this PR: - classifyGitFailure's hostname-mismatch regex missed curl's actual wording for an IP-address SAN mismatch, so the raw stderr leaked through instead of the classified TLS message. - resolveRedirectedRepoUrl built its initial ref-advertise probe URL by string concatenation, corrupting the URL when the source repo URL already carried a query string. - Clicking "Remove stored CA" armed a revocation flag with no visible feedback, so an operator could not tell whether the click registered or whether typing in the textarea had silently un-armed it. Adds regression tests for all three.
271 lines
9.2 KiB
TypeScript
271 lines
9.2 KiB
TypeScript
import { randomUUID } from 'crypto';
|
|
import path from 'path';
|
|
import { NodeRegistry } from '../NodeRegistry';
|
|
import { MANAGED_ROOT_NAME } from './managedPaths';
|
|
import { encodeGitOpsJson } from './json';
|
|
import { materializationFingerprint } from './fingerprint';
|
|
import { parseLegacyRepoUrl, parseStorableRepoUrl, secretFreeRepoUrl, secretFreeRepoUrlFromStorable, serializeRepoIdentity, serializeRepoIdentityFromStorable, type RepoIdentity } from './repoIdentity';
|
|
import type { RefKind } from '../git/types';
|
|
import type {
|
|
GitOpsApplicationRow,
|
|
GitOpsCreateCheckpointRow,
|
|
GitOpsGenerationRow,
|
|
} from './types';
|
|
|
|
/** The material source configuration a Direct application is bound to. */
|
|
export type DirectSourceConfig = {
|
|
repoUrl: string;
|
|
branch: string;
|
|
composePaths: readonly string[];
|
|
contextDir: string | null;
|
|
syncEnv: boolean;
|
|
envPath: string | null;
|
|
};
|
|
|
|
export type DirectSourceIdentity = {
|
|
/** Secret-free `https://host/pathname`, safe to persist and to project. */
|
|
repoUrl: string;
|
|
identity: RepoIdentity;
|
|
fingerprint: string;
|
|
};
|
|
|
|
export class GitOpsIdentityError extends Error {
|
|
constructor(message: string) {
|
|
super(message);
|
|
this.name = 'GitOpsIdentityError';
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Derive the storable identity and materialization fingerprint for a source.
|
|
*
|
|
* The fingerprint is what later decides whether a staged candidate still
|
|
* matches the configuration it was built from, so it is computed from the same
|
|
* secret-free identity that gets persisted, never from the raw operational URL.
|
|
*/
|
|
export function directSourceIdentity(config: DirectSourceConfig): DirectSourceIdentity {
|
|
const parsed = parseStorableRepoUrl(config.repoUrl);
|
|
if (!parsed.ok) throw new GitOpsIdentityError(`repository URL is not storable: ${parsed.reason}`);
|
|
const identity = serializeRepoIdentityFromStorable(parsed);
|
|
const repoUrl = secretFreeRepoUrlFromStorable(parsed);
|
|
const fingerprint = materializationFingerprint({
|
|
repoIdentity: identity,
|
|
configuredRef: config.branch,
|
|
composePaths: config.composePaths,
|
|
contextDir: config.contextDir,
|
|
syncEnv: config.syncEnv,
|
|
envPath: config.envPath,
|
|
});
|
|
return { repoUrl, identity, fingerprint };
|
|
}
|
|
|
|
/**
|
|
* The same derivation, for URLs that predate strict ingress.
|
|
*
|
|
* Migration is its only caller. A legacy operational row may still carry
|
|
* userinfo or a query string that fetch needs, so the storable identity strips
|
|
* them instead of refusing the stack; the strict helper above stays the gate
|
|
* for every path a user can drive.
|
|
*/
|
|
export function migrationDirectSourceIdentity(config: DirectSourceConfig): DirectSourceIdentity {
|
|
const parsed = parseLegacyRepoUrl(config.repoUrl);
|
|
if (!parsed.ok) throw new GitOpsIdentityError(`repository URL is not storable: ${parsed.reason}`);
|
|
return directSourceIdentityFromUrl(config, parsed.url);
|
|
}
|
|
|
|
function directSourceIdentityFromUrl(config: DirectSourceConfig, url: URL): DirectSourceIdentity {
|
|
const identity = serializeRepoIdentity(url);
|
|
const material = {
|
|
repoIdentity: identity,
|
|
configuredRef: config.branch,
|
|
composePaths: config.composePaths,
|
|
contextDir: config.contextDir,
|
|
syncEnv: config.syncEnv,
|
|
envPath: config.envPath,
|
|
};
|
|
return {
|
|
repoUrl: secretFreeRepoUrl(identity),
|
|
identity,
|
|
fingerprint: materializationFingerprint(material),
|
|
};
|
|
}
|
|
|
|
/** Absolute managed root for one stack on the local node. */
|
|
export function stackManagedRoot(stackName: string): string {
|
|
const dataDir = process.env.DATA_DIR || path.join(process.cwd(), 'data');
|
|
const nodeId = NodeRegistry.getInstance().getDefaultNodeId();
|
|
return path.join(dataDir, MANAGED_ROOT_NAME, String(nodeId), stackName);
|
|
}
|
|
|
|
export function newGitOpsId(): string {
|
|
return randomUUID();
|
|
}
|
|
|
|
/**
|
|
* Build a Direct application row.
|
|
*
|
|
* `creating` is for create-from-Git, where the stack does not exist yet and the
|
|
* checkpoint decides what happens if the process dies. `active` is for linking
|
|
* a stack that already exists: there is nothing to recover, so it is live from
|
|
* the moment the source row commits.
|
|
*/
|
|
export function buildDirectApplicationRow(args: {
|
|
id: string;
|
|
stackName: string;
|
|
config: DirectSourceConfig;
|
|
identity: DirectSourceIdentity;
|
|
lifecycleStatus: 'creating' | 'active';
|
|
at: number;
|
|
}): GitOpsApplicationRow {
|
|
return {
|
|
id: args.id,
|
|
lifecycle_key: `direct:${args.stackName}`,
|
|
lifecycle_status: args.lifecycleStatus,
|
|
target_mode: 'direct',
|
|
stack_name: args.stackName,
|
|
blueprint_id: null,
|
|
configured_repo_url: args.identity.repoUrl,
|
|
repo_identity_json: encodeGitOpsJson(args.identity.identity),
|
|
configured_ref: args.config.branch,
|
|
compose_paths_json: encodeGitOpsJson([...args.config.composePaths]),
|
|
context_dir: args.config.contextDir,
|
|
sync_env: args.config.syncEnv ? 1 : 0,
|
|
env_path: args.config.syncEnv ? args.config.envPath : null,
|
|
materialization_fingerprint: args.identity.fingerprint,
|
|
desired_commit_sha: null,
|
|
fetched_commit_sha: null,
|
|
fetched_resolved_ref_kind: null,
|
|
candidate_generation_id: null,
|
|
accepted_generation_id: null,
|
|
candidate_plan_blocked: 0,
|
|
review_required: 0,
|
|
artifact_set_id: null,
|
|
latest_artifact_set_id: null,
|
|
intent_revision_id: null,
|
|
rollout_candidate_id: null,
|
|
rollout_generation_id: null,
|
|
source_acceptance_ref: null,
|
|
placement_approval_ref: null,
|
|
rollout_authorization_ref: null,
|
|
legacy_combined_approval_ref: null,
|
|
preflight_fingerprint: null,
|
|
latest_operation_id: null,
|
|
active_operation_id: null,
|
|
active_operation_stage: null,
|
|
active_operation_at: null,
|
|
active_generation_id: null,
|
|
pause_at: null,
|
|
pause_reason: null,
|
|
partial_json: null,
|
|
failure_stage: null,
|
|
failure_class: null,
|
|
failure_at: null,
|
|
retry_at: null,
|
|
retry_count: 0,
|
|
suspended_at: null,
|
|
recovery_ref: null,
|
|
recovery_phase: null,
|
|
interruption_stage: null,
|
|
interruption_at: null,
|
|
interruption_operation_id: null,
|
|
interruption_generation_id: null,
|
|
evidence_fresh_at: null,
|
|
evidence_limitations_json: null,
|
|
created_at: args.at,
|
|
updated_at: args.at,
|
|
};
|
|
}
|
|
|
|
export function buildGenerationRow(args: {
|
|
id: string;
|
|
applicationId: string;
|
|
commitSha: string;
|
|
identity: DirectSourceIdentity;
|
|
configuredRef: string;
|
|
resolvedRefKind: RefKind;
|
|
candidateRelPath: string;
|
|
appliedRelPath: string;
|
|
manifestVersion: number;
|
|
expectedInvocation: unknown;
|
|
changePlanFingerprint: string | null;
|
|
operationId: string;
|
|
trigger: string;
|
|
actor: string | null;
|
|
at: number;
|
|
/** A blocked change plan is recorded, but such a generation can never apply. */
|
|
planBlocked?: boolean;
|
|
}): GitOpsGenerationRow {
|
|
return {
|
|
id: args.id,
|
|
application_id: args.applicationId,
|
|
commit_sha: args.commitSha,
|
|
repo_url: args.identity.repoUrl,
|
|
configured_ref: args.configuredRef,
|
|
resolved_ref_kind: args.resolvedRefKind,
|
|
repo_identity_json: encodeGitOpsJson(args.identity.identity),
|
|
manifest_version: args.manifestVersion,
|
|
candidate_dir: args.candidateRelPath,
|
|
applied_dir: args.appliedRelPath,
|
|
expected_invocation_json: encodeGitOpsJson(args.expectedInvocation),
|
|
materialization_fingerprint: args.identity.fingerprint,
|
|
validation_ok: 1,
|
|
plan_blocked: args.planBlocked ? 1 : 0,
|
|
change_plan_fingerprint: args.changePlanFingerprint,
|
|
operation_id: args.operationId,
|
|
trigger: args.trigger,
|
|
actor: args.actor,
|
|
previous_generation_id: null,
|
|
redacted_limitations_json: '[]',
|
|
created_at: args.at,
|
|
};
|
|
}
|
|
|
|
export function buildCreateCheckpointRow(args: {
|
|
applicationId: string;
|
|
stackName: string;
|
|
operationId: string;
|
|
config: DirectSourceConfig;
|
|
identity: DirectSourceIdentity;
|
|
authType: string;
|
|
encryptedToken: string | null;
|
|
encryptedDeployKey?: string | null;
|
|
sshKnownHostsEntry?: string | null;
|
|
sshHostKeyFingerprint?: string | null;
|
|
encryptedCaBundle?: string | null;
|
|
autoApplyOnWebhook: boolean;
|
|
autoDeployOnApply: boolean;
|
|
commitSha: string;
|
|
createdManagedRoot: boolean;
|
|
at: number;
|
|
}): GitOpsCreateCheckpointRow {
|
|
return {
|
|
application_id: args.applicationId,
|
|
stack_name: args.stackName,
|
|
phase: 'pre_stack',
|
|
generation_id: null,
|
|
operation_id: args.operationId,
|
|
// Operational URL for fetch compatibility during create. Copies into
|
|
// generations and history always go through the secret-free identity.
|
|
repo_url: args.config.repoUrl,
|
|
branch: args.config.branch,
|
|
compose_path: args.config.composePaths[0] ?? '',
|
|
compose_paths_json: encodeGitOpsJson([...args.config.composePaths]),
|
|
context_dir: args.config.contextDir,
|
|
sync_env: args.config.syncEnv ? 1 : 0,
|
|
env_path: args.config.syncEnv ? args.config.envPath : null,
|
|
auth_type: args.authType,
|
|
encrypted_token: args.encryptedToken,
|
|
encrypted_deploy_key: args.encryptedDeployKey ?? null,
|
|
ssh_known_hosts_entry: args.sshKnownHostsEntry ?? null,
|
|
ssh_host_key_fingerprint: args.sshHostKeyFingerprint ?? null,
|
|
encrypted_ca_bundle: args.encryptedCaBundle ?? null,
|
|
auto_apply_on_webhook: args.autoApplyOnWebhook ? 1 : 0,
|
|
auto_deploy_on_apply: args.autoDeployOnApply ? 1 : 0,
|
|
commit_sha: args.commitSha,
|
|
applied_spec_json: null,
|
|
created_managed_root: args.createdManagedRoot ? 1 : 0,
|
|
created_at: args.at,
|
|
updated_at: args.at,
|
|
};
|
|
}
|