Files
Sebastian Poxhofer 4b4d14ec00 feat(manager): pass extracted constraints to updateArtifacts (#45826)
* feat(manager): pass extracted constraints to updateArtifacts

`extractedConstraints` were collected during extraction but only ever read
by `fetch.ts` for the lookup. At artifact time every manager had to derive
the same tool constraints a second time, and when the updated package file
no longer carried the constraint there was nothing to fall back to.

`generateBranchConfig()` now merges the `extractedConstraints` of all
upgrades of a branch onto the branch config, which reaches `updateArtifacts()`
through `patchConfigForArtifactsUpdate()`. `UpdateArtifactsConfig` declares
the field and documents what it contains.

The new `resolveToolConstraint()` helper implements the ladder once:
user `constraints` > an optional artifact-time derivation > the extracted
value. The derivation is ranked above the extracted value on purpose: it
reads the package files as the branch changed them, so poetry (python is an
updatable dependency there), composer, pipenv (`pipenv` can be a dev
package) and pep621 (`requires-python` is an updatable dependency) would
otherwise regress to a stale constraint. pipenv, pep621 uv/pdm, poetry and
composer are migrated to the helper and keep their derivation.

Behaviour change: these managers now use the extract-time constraint as a
last resort instead of running unconstrained, for example on lock file
maintenance where pipenv deletes `Pipfile.lock` before deriving.

`execute-post-upgrade-commands.ts` is unaffected: it reads
`upgrade.constraints` per upgrade, and the extracted values are kept in a
separate field rather than folded into `constraints`, so post-upgrade task
tool constraints still come from user config only.

Not migrated, because their constraints are never extracted, so the
fallback would be dead code: bundler (`ruby`/`bundler`), pixi (`pixi`),
pep621 uv (`required-version`) and npm's node ladder, whose last step
already reads `package.json` engines.

* refactor(manager): resolve tool constraints through resolveToolConstraint()

Every manager now obtains the constraint of a tool it runs through
`resolveToolConstraint()` instead of reading `config.constraints` itself, so
the precedence "user config, then a value derived from the updated files, then
the constraints collected during extraction" applies everywhere. The helper
treats an empty string as not set at every step and accepts the
`PostUpdateConfig` shape, whose `constraints` may be `null`.

Behaviour changes that go beyond the new fallback to extracted constraints:

- `mix` applies its hardcoded erlang default only after the extracted
  constraint, so an erlang version collected at extraction can win over it.
- `kustomize` decides `HELM_EXPERIMENTAL_OCI` from the fully resolved helm
  constraint; `generateHelmEnvs()` takes that value instead of the config.
- `npm` (yarn): an upgrade of yarn without a new value now consults the user
  constraint before deriving one from package.json or the lock file.
- The "Using <tool> constraint from config" debug lines in bundler, composer,
  copier and pip-compile are gone, as the helper is the only reader of the
  config now.

Helpers that read the config synchronously became async: bundler
`getBundlerConstraint()`, copier `getPythonVersionConstraint()` and
`getCopierVersionConstraint()`, pip-compile `get*VersionConstraint()` and
`getToolVersionConstraint()`, gomod `deriveGoToolchainConstraints()` and
`getUpdateImportPathCmds()`, mise `getMiseLockToolConstraints()`. composer
`getPhpConstraint()` is removed.

Each converted manager has a test showing the extracted constraint reaching
the executed tool, and, where a derive callback exists, that the derived value
still wins over it.

* refactor(linters): add prefer-resolve-tool-constraint rule

Reports member reads and destructuring of `constraints` and
`extractedConstraints` from a manager config in `lib/modules/manager/**`, so
that tool constraints are resolved through `resolveToolConstraint()` in
`lib/modules/manager/util.ts` only. Writes and the same properties on package
files, which extraction has to populate, are not reported. The helper itself
and spec files are exempt in `.oxlintrc.json`.

* refactor(manager): inline thin resolveToolConstraint wrappers

Drop copier/utils.ts and the pip-compile getPipToolsVersionConstraint /
getUvVersionConstraint helpers, which only forwarded to
resolveToolConstraint(). Callers now call it directly.

* docs(development): note lock file maintenance in tool constraints section

* refactor(manager): address review on tool constraint resolution

composer only runs extractConstraints() when a user constraint is missing,
via Lazy. UpdateArtifactsConfig no longer re-declares constraints and
extractedConstraints, which ToolConstraintsConfig already provides.
2026-09-14 11:09:07 +00:00

535 lines
15 KiB
TypeScript

import type { Stats } from 'node:fs';
import os from 'node:os';
import { isTruthy } from '@sindresorhus/is';
import { quote } from 'shlex';
import upath from 'upath';
import { GlobalConfig } from '../../../config/global.ts';
import { logger } from '../../../logger/index.ts';
import { withCache } from '../../../util/cache/package/with-cache.ts';
import { exec } from '../../../util/exec/index.ts';
import type { ExecOptions, ExtraEnv } from '../../../util/exec/types.ts';
import {
chmodLocalFile,
deleteLocalFile,
readLocalFile,
statLocalFile,
writeLocalFile,
} from '../../../util/fs/index.ts';
import { getRepoStatus } from '../../../util/git/index.ts';
import type { StatusResult } from '../../../util/git/types.ts';
import { hashStream } from '../../../util/hash.ts';
import { Http } from '../../../util/http/index.ts';
import { regEx } from '../../../util/regex.ts';
import mavenVersioning from '../../versioning/maven/index.ts';
import type {
PackageDependency,
UpdateArtifact,
UpdateArtifactsConfig,
UpdateArtifactsResult,
} from '../types.ts';
import { resolveToolConstraint } from '../util.ts';
const http = new Http('maven-wrapper');
const DEFAULT_MAVEN_REPO_URL = 'https://repo.maven.apache.org/maven2';
function getChecksumFromUrl(url: string): Promise<string> {
return withCache(
{
namespace: 'url-sha256',
key: url,
ttlMinutes: 3 * 24 * 60, // 3 days
},
() => hashStream(http.stream(url), 'sha256'),
);
}
function getDistributionUrl(content: string): string | null {
const match = regEx(/^distributionUrl\s*=\s*(?<distributionUrl>.+)$/m).exec(
content,
);
return (
match?.groups?.distributionUrl?.replace(regEx(/\\:/g), ':').trim() ?? null
);
}
function getWrapperUrl(content: string): string | null {
const match = regEx(/^wrapperUrl\s*=\s*(?<wrapperUrl>.+)$/m).exec(content);
return match?.groups?.wrapperUrl?.replace(regEx(/\\:/g), ':').trim() ?? null;
}
function constructWrapperUrl(
version: string,
repoUrl: string = DEFAULT_MAVEN_REPO_URL,
): string {
return `${repoUrl}/org/apache/maven/wrapper/maven-wrapper/${version}/maven-wrapper-${version}.jar`;
}
function getWrapperVersion(content: string): string | null {
const match = regEx(/^wrapperVersion\s*=\s*(?<wrapperVersion>.+)$/m).exec(
content,
);
return match?.groups?.wrapperVersion?.trim() ?? null;
}
function getDistributionType(content: string): string | null {
const match = regEx(/^distributionType\s*=\s*(?<distributionType>.+)$/m).exec(
content,
);
return match?.groups?.distributionType?.trim() ?? null;
}
function getChecksumValue(content: string, key: string): string | null {
const match = regEx(`^${key}\\s*=\\s*(?<value>.+)$`, 'm').exec(content);
return match?.groups?.value?.trim() ?? null;
}
function addChecksumAfterLine(
content: string,
lineRegex: RegExp,
key: string,
value: string,
): string {
return content.replace(
regEx(lineRegex),
(lineMatch) => `${lineMatch}\n${key}=${value}`,
);
}
async function updateChecksums(
content: string,
updatedDeps: PackageDependency[],
addDistributionChecksum: boolean,
addWrapperChecksum: boolean,
originalContent: string,
): Promise<string> {
let updatedContent = content;
// Update or add distribution checksum
if (addDistributionChecksum) {
const existingChecksum = getChecksumValue(
updatedContent,
'distributionSha256Sum',
);
const fallbackChecksum = getChecksumValue(
originalContent,
'distributionSha256Sum',
);
const distUrl = getDistributionUrl(updatedContent);
if (distUrl) {
try {
const checksum = await getChecksumFromUrl(distUrl);
if (existingChecksum) {
updatedContent = updatedContent.replace(
regEx(/distributionSha256Sum=.*/),
`distributionSha256Sum=${checksum}`,
);
} else {
// Add checksum after distributionUrl
updatedContent = addChecksumAfterLine(
updatedContent,
regEx(/^(?:distributionUrl\s*=\s*.+)$/m),
'distributionSha256Sum',
checksum,
);
}
} catch (err) {
logger.warn(
{ err, url: distUrl },
'Failed to fetch distribution checksum',
);
if (!existingChecksum && fallbackChecksum) {
updatedContent = addChecksumAfterLine(
updatedContent,
regEx(/^(?:distributionUrl\s*=\s*.+)$/m),
'distributionSha256Sum',
fallbackChecksum,
);
}
}
}
}
// Update or add wrapper checksum
if (addWrapperChecksum) {
const existingChecksum = getChecksumValue(
updatedContent,
'wrapperSha256Sum',
);
const fallbackChecksum = getChecksumValue(
originalContent,
'wrapperSha256Sum',
);
let wrapperUrl = getWrapperUrl(updatedContent);
// If no wrapperUrl, try to construct from wrapperVersion
if (!wrapperUrl) {
const wrapperVersion = getWrapperVersion(updatedContent);
if (wrapperVersion) {
const customRepoUrl = getCustomMavenWrapperRepoUrl(updatedDeps);
wrapperUrl = constructWrapperUrl(
wrapperVersion,
customRepoUrl ?? DEFAULT_MAVEN_REPO_URL,
);
}
}
if (wrapperUrl) {
try {
const checksum = await getChecksumFromUrl(wrapperUrl);
if (existingChecksum) {
updatedContent = updatedContent.replace(
regEx(/wrapperSha256Sum=.*/),
`wrapperSha256Sum=${checksum}`,
);
} else {
// Add checksum after wrapperUrl or wrapperVersion
updatedContent = addChecksumAfterLine(
updatedContent,
regEx(/^(?:wrapperUrl\s*=\s*.+|wrapperVersion\s*=\s*.+)$/m),
'wrapperSha256Sum',
checksum,
);
}
} catch (err) {
logger.warn(
{ err, url: wrapperUrl },
'Failed to fetch wrapper checksum',
);
if (!existingChecksum && fallbackChecksum) {
updatedContent = addChecksumAfterLine(
updatedContent,
regEx(/^(?:wrapperUrl\s*=\s*.+|wrapperVersion\s*=\s*.+)$/m),
'wrapperSha256Sum',
fallbackChecksum,
);
}
}
}
}
return updatedContent;
}
interface MavenWrapperPaths {
wrapperExecutableFileName: string;
localProjectDir: string;
wrapperFullyQualifiedPath: string;
}
async function addIfUpdated(
status: StatusResult,
fileProjectPath: string,
): Promise<UpdateArtifactsResult | null> {
if (status.modified.includes(fileProjectPath)) {
return {
file: {
type: 'addition',
path: fileProjectPath,
contents: await readLocalFile(fileProjectPath),
},
};
}
return null;
}
export async function updateArtifacts({
packageFileName,
newPackageFileContent,
updatedDeps,
config,
}: UpdateArtifact): Promise<UpdateArtifactsResult[] | null> {
try {
logger.debug({ updatedDeps }, 'maven-wrapper.updateArtifacts()');
const hasMavenUpdate = updatedDeps.some((dep) => dep.depName === 'maven');
const hasWrapperUpdate = updatedDeps.some(
(dep) => dep.depName === 'maven-wrapper',
);
// Remember original content properties - we'll restore them after wrapper:wrapper
const hadDistributionChecksum = newPackageFileContent.includes(
'distributionSha256Sum=',
);
const hadWrapperChecksum =
newPackageFileContent.includes('wrapperSha256Sum=');
const originalDistributionType = getDistributionType(newPackageFileContent);
// Skip if no relevant updates
if (!hasWrapperUpdate && !(hasMavenUpdate && hadDistributionChecksum)) {
logger.debug(
'No Maven wrapper or distribution checksum updates - skipping Artifacts update',
);
return null;
}
// If wrapper is being updated, check if mvnw exists first
let cmd: string | null = null;
if (hasWrapperUpdate) {
cmd = await createWrapperCommand(
packageFileName,
originalDistributionType,
);
if (!cmd) {
logger.info('No mvnw found - skipping Artifacts update');
return null;
}
}
// Delete old maven-wrapper.jar if checksums exist and wrapper:wrapper will run
// This prevents checksum validation failure when wrapper:wrapper runs
if (hasWrapperUpdate && (hadWrapperChecksum || hadDistributionChecksum)) {
const jarPath = packageFileName.replace(
'maven-wrapper.properties',
'maven-wrapper.jar',
);
if (jarPath.endsWith('maven-wrapper.jar')) {
try {
await deleteLocalFile(jarPath);
logger.debug({ jarPath }, 'Deleted old maven-wrapper.jar');
} catch {
// File may not exist, ignore
}
}
}
// Strip checksum lines before running wrapper:wrapper to avoid validation failure
// (mvnw would try to validate NEW jar against OLD checksum)
let contentToWrite = newPackageFileContent;
if (hasWrapperUpdate && (hadWrapperChecksum || hadDistributionChecksum)) {
contentToWrite = contentToWrite
.replace(regEx(/^distributionSha256Sum=.*\n?/m), '')
.replace(regEx(/^wrapperSha256Sum=.*\n?/m), '');
}
// Write the properties file
await writeLocalFile(packageFileName, contentToWrite);
// Run wrapper:wrapper if the wrapper itself is being updated
if (hasWrapperUpdate && cmd) {
const extraEnv = getExtraEnvOptions(updatedDeps);
await executeWrapperCommand(cmd, config, packageFileName, extraEnv);
}
// Now update checksums AFTER wrapper:wrapper has run (if it ran)
// This ensures we compute checksums for the actual generated content
if (hadDistributionChecksum || hadWrapperChecksum) {
const currentContent = await readLocalFile(packageFileName, 'utf8');
if (currentContent) {
const contentWithChecksums = await updateChecksums(
currentContent,
updatedDeps,
hadDistributionChecksum,
hadWrapperChecksum,
newPackageFileContent,
);
await writeLocalFile(packageFileName, contentWithChecksums);
}
}
const status = await getRepoStatus();
const artifactFileNames = [
'.mvn/wrapper/maven-wrapper.properties',
'.mvn/wrapper/maven-wrapper.jar',
'.mvn/wrapper/MavenWrapperDownloader.java',
'mvnw',
'mvnw.cmd',
].map(
(filename) =>
packageFileName.replace('.mvn/wrapper/maven-wrapper.properties', '') +
filename,
);
const updateArtifactsResult = (
await getUpdatedArtifacts(status, artifactFileNames)
).filter(isTruthy);
logger.debug(
{ files: updateArtifactsResult.map((r) => r.file?.path) },
`Returning updated maven-wrapper files`,
);
return updateArtifactsResult;
} catch (err) {
logger.debug({ err }, 'Error setting new Maven Wrapper release value');
return [
{
artifactError: {
fileName: packageFileName,
stderr: err.message,
},
},
];
}
}
async function getUpdatedArtifacts(
status: StatusResult,
artifactFileNames: string[],
): Promise<UpdateArtifactsResult[]> {
const updatedResults: UpdateArtifactsResult[] = [];
for (const artifactFileName of artifactFileNames) {
const updatedResult = await addIfUpdated(status, artifactFileName);
if (updatedResult !== null) {
updatedResults.push(updatedResult);
}
}
return updatedResults;
}
/**
* Find compatible java version for maven.
* see https://maven.apache.org/developers/compatibility-plan.html
* @param mavenWrapperVersion current maven version
* @returns A Java semver range
*/
export function getJavaConstraint(
mavenWrapperVersion: string | null | undefined,
): string | null {
const major = mavenWrapperVersion
? mavenVersioning.getMajor(mavenWrapperVersion)
: null;
if (major && major >= 3) {
return '^17.0.0';
}
return '^8.0.0';
}
async function executeWrapperCommand(
cmd: string,
config: UpdateArtifactsConfig,
packageFileName: string,
extraEnv: ExtraEnv,
): Promise<void> {
logger.debug(`Updating maven wrapper: "${cmd}"`);
const { wrapperFullyQualifiedPath } = getMavenPaths(packageFileName);
const execOptions: ExecOptions = {
cwdFile: wrapperFullyQualifiedPath,
docker: {},
extraEnv,
toolConstraints: [
{
toolName: 'java',
constraint: await resolveToolConstraint(config, 'java', () =>
getJavaConstraint(config.currentValue),
),
},
],
};
try {
await exec(cmd, execOptions);
} catch (err) {
logger.error({ err }, 'Error executing maven wrapper update command.');
throw err;
}
}
function getExtraEnvOptions(deps: PackageDependency[]): ExtraEnv {
const customMavenWrapperUrl = getCustomMavenWrapperRepoUrl(deps);
if (customMavenWrapperUrl) {
return { MVNW_REPOURL: customMavenWrapperUrl };
}
return {};
}
function getCustomMavenWrapperRepoUrl(
deps: PackageDependency[],
): string | null {
const replaceString = deps.find(
(dep) => dep.depName === 'maven-wrapper',
)?.replaceString;
if (!replaceString) {
return null;
}
const match = regEx(/^(?<repoUrl>.*?)\/org\/apache\/maven\/wrapper\//).exec(
replaceString,
);
if (!match) {
return null;
}
return match.groups!.repoUrl === DEFAULT_MAVEN_REPO_URL
? null
: match.groups!.repoUrl;
}
async function createWrapperCommand(
packageFileName: string,
distributionType: string | null,
): Promise<string | null> {
const {
wrapperExecutableFileName,
localProjectDir,
wrapperFullyQualifiedPath,
} = getMavenPaths(packageFileName);
// Use existing distributionType or default to 'script' to preserve JAR-based mode
// (prevents Maven 3.3.x from defaulting to only-script which doesn't support checksums)
const type = distributionType ?? 'script';
const args = `wrapper:wrapper -Dtype=${quote(type)}`;
return await prepareCommand(
wrapperExecutableFileName,
localProjectDir,
await statLocalFile(wrapperFullyQualifiedPath),
args,
);
}
function mavenWrapperFileName(): string {
if (
os.platform() === 'win32' &&
GlobalConfig.get('binarySource') !== 'docker'
) {
return 'mvnw.cmd';
}
return './mvnw';
}
function getMavenPaths(packageFileName: string): MavenWrapperPaths {
const wrapperExecutableFileName = mavenWrapperFileName();
const localProjectDir = upath.join(
upath.dirname(packageFileName),
packageFileName.includes('mvnw') ? '.' : '../../',
);
const wrapperFullyQualifiedPath = upath.join(
localProjectDir,
wrapperExecutableFileName,
);
return {
wrapperExecutableFileName,
localProjectDir,
wrapperFullyQualifiedPath,
};
}
async function prepareCommand(
fileName: string,
cwd: string | undefined,
pathFileStats: Stats | null,
args: string | null,
): Promise<string | null> {
/* v8 ignore next -- hard to test */
if (pathFileStats?.isFile() === true) {
// if the file is not executable by others
if (os.platform() !== 'win32' && (pathFileStats.mode & 0o1) === 0) {
// add the execution permission to the owner, group and others
logger.warn('Maven wrapper is missing the executable bit');
await chmodLocalFile(
// TODO: types (#22198)
upath.join(cwd!, fileName),
pathFileStats.mode | 0o111,
);
}
if (args === null) {
return fileName;
}
return `${fileName} ${args}`;
}
return null;
}