Files
Sebastian Poxhofer a1971afa78 fix(linters): add linting rule to enforce usage of our regEx utility (#44690)
* ci(linter): add renovate/require-regex-util rule

Adds a custom oxlint rule that flags raw RegExp usage in lib/
production code: any `new RegExp(...)` call and any regex literal
that is not the direct first argument of `regEx(...)`. The regEx()
helper from lib/util/regex.ts uses the RE2 engine for ReDoS-safe
matching and caches compiled patterns. lib/util/regex.ts itself is
exempt.

The rule is disabled for **/*.spec.ts, test/**, and tools/** via
config overrides, because vitest APIs such as expect.stringMatching
need bare RegExp instances (a past commit reverted regEx() there).

Severity is set to "warn" instead of "error": the repo currently has
197 pre-existing violations (above the agreed mechanical-fix
threshold of 80), and a number of them are deliberately
RE2-incompatible patterns (lookahead/lookbehind/named backreferences,
see TODO #12872/#12875 markers in e.g. lib/modules/versioning/rez/
pattern.ts and lib/modules/manager/dockerfile/extract.ts) that need
per-site judgment before they can be wrapped in regEx() or suppressed
with a justified disable comment. No violation fixes are included in
this commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci(linter): enforce renovate/require-regex-util at error

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: use regEx() for regex literals and RegExp construction in lib

Burn down all renovate/require-regex-util violations (197):

- wrap RE2-compatible regex literals and RegExp constructions in regEx()
- rewrite lookaround patterns where behavior is provably preserved:
  goproxy separator split, sbt-package link extraction, leiningen
  comment stripping and keyword search, npm GitHub owner validation
  (unit-count bound replaced by an explicit length check)
- keep native RegExp with disable comments where RE2 cannot be used:
  moo / good-enough-parser lexer patterns (moo rejects RE2 instances),
  MigrationsService propertyName (matched via isRegExp, which RE2
  instances fail), rez named backreferences and lookaheads, GitHub
  markdown link negative lookbehind, emoji astral-plane matching
- drop stale TODO #12875 comments for patterns current re2 handles

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: reduce require-regex-util disable comments

Rewrite the github massage-markdown-links URL regex to capture an
optional `api.` prefix in a named group (skipped in code) instead of
relying on a negative lookbehind, so both the base and global variants
can use regEx()/RE2 and their two disable comments are removed. Add
spec cases for api-prefixed URLs that match the URL shape.

Consolidate the four related line disables in rez/pattern.ts (bound and
range patterns emulating Python conditional patterns via named
backreferences and lookaheads, which RE2 cannot express) into a single
block disable/enable pair, matching the treatment of the lexer files.

Disable comment count drops from 13 to 8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(datasource/sbt-package): use named capture group in link regex

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(manager/cake): restore original multiLineComment regex spelling

The [^] -> [\\s\\S] rewrite was only needed for an RE2 wrap that was
later reverted; keep the diff minimal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(datasource/go): skip empty GOPROXY segments

The switch from a lookahead split to a capture-group split (needed because
RE2 has no lookahead support) changed how empty GOPROXY segments parse:
`a|,b` yielded `{ url: ',', fallback: '|' }` before and
`{ url: '', fallback: ',' }` after. The comma fallback only continues on
404/410, so a failing proxy ahead of an empty segment silently stopped the
lookup instead of falling through to the remaining proxies.

Empty segments carry no url to query, so drop them.

* refactor(datasource/go): use isTruthy for empty segment filter

* fix(datasource/go): filter empty segments before mapping

Filtering after the map broke the contextual typing from the
GoproxyItem[] annotation, widening fallback to string.

* refactor(platform/github): use regEx() for merge queue query stripping

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 14:30:46 +00:00

257 lines
7.5 KiB
TypeScript

import { lang, query as q } from '@renovatebot/good-enough-parser';
import { isTruthy } from '@sindresorhus/is';
import { quote } from 'shlex';
import upath from 'upath';
import { TEMPORARY_ERROR } from '../../../constants/error-messages.ts';
import { logger } from '../../../logger/index.ts';
import {
exec,
getToolSettingsOptions,
gradleJvmArg,
} from '../../../util/exec/index.ts';
import type { ExecOptions } from '../../../util/exec/types.ts';
import {
localPathExists,
readLocalFile,
writeLocalFile,
} from '../../../util/fs/index.ts';
import { getRepoStatus } from '../../../util/git/index.ts';
import type { StatusResult } from '../../../util/git/types.ts';
import { Http } from '../../../util/http/index.ts';
import { newlineRegex, regEx } from '../../../util/regex.ts';
import { replaceAt } from '../../../util/string.ts';
import { isGradleExecutionAllowed } from '../gradle/artifacts.ts';
import { updateArtifacts as gradleUpdateArtifacts } from '../gradle/index.ts';
import type {
UpdateArtifact,
UpdateArtifactsConfig,
UpdateArtifactsResult,
} from '../types.ts';
import {
extraEnv,
getJavaConstraint,
gradleWrapperFileName,
prepareGradleCommand,
} from './utils.ts';
const http = new Http('gradle-wrapper');
const groovy = lang.createLang('groovy');
type Ctx = string[];
async function addIfUpdated(
status: StatusResult,
fileProjectPath: string,
): Promise<UpdateArtifactsResult | null> {
if (status.modified.includes(fileProjectPath)) {
return {
file: {
type: 'addition',
path: fileProjectPath,
contents: await readLocalFile(fileProjectPath),
},
};
}
return null;
}
function getDistributionUrl(newPackageFileContent: string): string | null {
const distributionUrlLine = newPackageFileContent
.split(newlineRegex)
.find((line) => line.startsWith('distributionUrl='));
if (distributionUrlLine) {
return distributionUrlLine
.replace('distributionUrl=', '')
.replace('https\\:', 'https:');
}
return null;
}
async function getDistributionChecksum(url: string): Promise<string> {
const { body } = await http.getText(`${url}.sha256`);
return body;
}
export async function updateBuildFile(
localGradleDir: string,
wrapperProperties: Record<string, string | undefined | null>,
): Promise<string> {
let buildFileName = upath.join(localGradleDir, 'build.gradle');
if (!(await localPathExists(buildFileName))) {
buildFileName = upath.join(localGradleDir, 'build.gradle.kts');
}
const buildFileContent = await readLocalFile(buildFileName, 'utf8');
if (!buildFileContent) {
logger.debug('build.gradle or build.gradle.kts not found');
return buildFileName;
}
let buildFileUpdated = buildFileContent;
for (const [propertyName, newValue] of Object.entries(wrapperProperties)) {
if (!newValue) {
continue;
}
const query = q.tree({
type: 'wrapped-tree',
maxDepth: 1,
search: q
.sym<Ctx>(propertyName)
.op('=')
.str((ctx, { value, offset }) => {
buildFileUpdated = replaceAt(
buildFileUpdated,
offset,
value,
newValue,
);
return ctx;
}),
});
groovy.query(buildFileUpdated, query, []);
}
await writeLocalFile(buildFileName, buildFileUpdated);
return buildFileName;
}
export async function updateLockFiles(
buildFileName: string,
config: UpdateArtifactsConfig,
): Promise<UpdateArtifactsResult[] | null> {
const buildFileContent = await readLocalFile(buildFileName, 'utf8');
if (!buildFileContent) {
logger.debug('build.gradle or build.gradle.kts not found');
return null;
}
return await gradleUpdateArtifacts({
packageFileName: buildFileName,
updatedDeps: [],
newPackageFileContent: buildFileContent,
config,
});
}
export async function updateArtifacts({
packageFileName,
newPackageFileContent,
updatedDeps,
config,
}: UpdateArtifact): Promise<UpdateArtifactsResult[] | null> {
try {
logger.debug({ updatedDeps }, 'gradle-wrapper.updateArtifacts()');
const localGradleDir = upath.join(upath.dirname(packageFileName), '../../');
const gradlewFile = upath.join(localGradleDir, gradleWrapperFileName());
let cmd = await prepareGradleCommand(gradlewFile);
if (!cmd) {
logger.info('No gradlew found - skipping Artifacts update');
return null;
}
if (!isGradleExecutionAllowed(gradlewFile)) {
logger.trace(
'Not allowed to execute gradle due to allowedUnsafeExecutions - aborting update',
);
return null;
}
// Limit the Gradle daemon Java heap memory size to prevent OOM errors
// leading to Renovate kernel-OOMs and timeouts. See #39558
cmd += gradleJvmArg(getToolSettingsOptions(config.toolSettings));
cmd += ' :wrapper';
let checksum: string | null = null;
const distributionUrl = getDistributionUrl(newPackageFileContent);
if (distributionUrl) {
cmd += ` --gradle-distribution-url ${quote(distributionUrl)}`;
if (newPackageFileContent.includes('distributionSha256Sum=')) {
//update checksum in case of distributionSha256Sum in properties then run wrapper
checksum = await getDistributionChecksum(distributionUrl);
await writeLocalFile(
packageFileName,
newPackageFileContent.replace(
regEx(/distributionSha256Sum=.*/),
`distributionSha256Sum=${checksum}`,
),
);
cmd += ` --gradle-distribution-sha256-sum ${quote(checksum)}`;
}
} else {
cmd += ` --gradle-version ${quote(config.newValue!)}`;
}
logger.debug(`Updating gradle wrapper: "${cmd}"`);
const execOptions: ExecOptions = {
cwdFile: gradlewFile,
docker: {},
extraEnv,
toolConstraints: [
{
toolName: 'java',
constraint:
config.constraints?.java ??
(await getJavaConstraint(config.currentValue, gradlewFile)),
},
],
};
try {
await exec(cmd, execOptions);
} catch (err) {
// istanbul ignore if
if (err.message === TEMPORARY_ERROR) {
throw err;
}
logger.warn(
{ err },
'Error executing gradle wrapper update command. This may not necessarily be a blocker to the update, so please verify with the gradle wrapper output logs.',
);
}
const buildFileName = await updateBuildFile(localGradleDir, {
gradleVersion: config.newValue,
distributionSha256Sum: checksum,
distributionUrl,
});
const lockFiles = await updateLockFiles(buildFileName, config);
const status = await getRepoStatus();
const artifactFileNames = [
packageFileName,
buildFileName,
...['gradle/wrapper/gradle-wrapper.jar', 'gradlew', 'gradlew.bat'].map(
(filename) => upath.join(localGradleDir, filename),
),
];
const updateArtifactsResult = (
await Promise.all(
artifactFileNames.map((fileProjectPath) =>
addIfUpdated(status, fileProjectPath),
),
)
).filter(isTruthy);
if (lockFiles) {
updateArtifactsResult.push(...lockFiles);
}
logger.debug(
{ files: updateArtifactsResult.map((r) => r.file?.path) },
`Returning updated gradle-wrapper files`,
);
return updateArtifactsResult;
} catch (err) {
logger.debug({ err }, 'Error setting new Gradle Wrapper release value');
return [
{
artifactError: {
fileName: packageFileName,
stderr: err.message,
},
},
];
}
}