When running Renovate with `binarySource=install` on a repository with
many branches, the amount of time running `install-tool` - on already
installed tools - can take a significant percentage of the overall run.
For instance, on one repository we have instrumented with OpenTelemetry,
we see:
- initial `install-tool golang`: 8.7s
- follow-up `install-tool golang`: median 635ms, max 2.4s
However, because we run `install-tool golang` 640 times, this adds up to
24.8/29.2m (85%) of the repo's run time, which is an overwhelming
percentage when we're not even re-installing the tool.
To handle this more effectively, we can add an in-memory cache of all
the tools (and resolved versions) that we've installed in a given run.
We can't do this with `binarySource=docker`, as it has a fresh container
each time.
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* refactor(linters): add prefer-coerce-array rule
Refs #45217
`x ?? []` and `coerceArray(x)` say the same thing, but only the second
one is searchable and consistent. The rule flags a `??` whose default is
a bare `[]` literal; a non-empty default or an asserted `([] as Foo[])`
still passes.
Applying it across `lib/` surfaced three spots the helper cannot cover,
which keep `?? []` with a disable comment or a narrower fix:
- `package-files.ts` destructures a tuple, which `coerceArray()` widens
- `generate.spec.ts` defaults `matchAll()`, an iterator rather than an array
- `mise/upgradeable-tooling.ts` declares a `Record` return, so its
`?? []` fallback was wrong to begin with and becomes `coerceObject()`
* refactor(platform/github): hoist the GHE version header lookup
`Object.entries(...).find()` returns a tuple, and defaulting it with
`?? []` only to destructure the second element reads as an array
default when it is not one. Name the found entry instead, which drops
the `?? []` rather than routing a tuple through `coerceArray()`.
As noted in #43761, we had recently set `CI=true` for the Renovate and
Containerbase orgs on Mend-hosted apps due to a pnpm v11 requirement:
ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY Aborted removal of
modules directory due to no TTY
If you are running pnpm in CI, set the CI environment variable to
"true", or set "confirmModulesPurge" to "false".
Similarly in #43606, it was noted that we should do this for `mise lock`
to mark the workspace as "trusted".
Although we went with a different situation, there will be cases where
this is useful for tools being executed, for instance when we invoke `yarn`.
We also make sure to keep our documentation up-to-date as we add to the
specific overrides.
With help from Claude Sonnet and Opus to:
- perform some of the mechanical updates as part of this change
- ensure that the Docker sidecar always has `hardcodedProcessEnv` passed
into it
- catch a needed change to `exposeAllEnv` handling
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* feat(lint): add renovate/prefer-is-helpers and renovate/prefer-is-object rules
Flag ad-hoc type checks that have @sindresorhus/is equivalents:
- .filter(Boolean) -> .filter(isTruthy)
- typeof x === 'string' -> isString(x)
- x === null || x === undefined -> isNullOrUndefined(x) (and negation)
The typeof x === 'object' shape lives in a separate suggestion-only rule
(renovate/prefer-is-object, kept at warn) because isObject()/isPlainObject()
are not drop-in equivalents: null, arrays and functions behave differently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: fix renovate/prefer-is-helpers violations
Replace ad-hoc type checks with @sindresorhus/is helpers across lib/:
- 13x .filter(Boolean) -> .filter(isTruthy)
- 36x typeof x ==/!== 'string' -> isString(x) / !isString(x)
- 6x x === null || x === undefined -> isNullOrUndefined(x) (and negation)
Promote renovate/prefer-is-helpers to error now that the codebase is clean.
renovate/prefer-is-object stays at warn: its 15 remaining sites guard
parsed YAML/JSON or unknown values where isObject()/isPlainObject() are
not provably equivalent (null/array/function semantics).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: enforce renovate/prefer-is-object at error level
Resolve the 15 remaining warnings and raise the rule from warn to error:
- Migrate 10 sites to isObject() where the guarded value provably cannot
be a function (parsed YAML/JSON, typed exec command arguments, or
test-controlled fixtures): helm-values, woodpecker, bun, pre-commit
managers, exec utils, file package cache, and the github http spec.
In the bun manager this also fixes a latent crash where a null
`workspaces` value would throw on the `in` operator.
- Keep 5 sites as reasoned oxlint-disable comments where excluding
functions is load-bearing: the byte-exact fingerprint canonicalizer
(2), the circularity walker, the zod result guard, and the generic
hasKey helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
As part of future changes, it makes it easier to work with the resulting
types, because it's clear when they are optional/required.
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
As part of future changes, we'll add options that do not require the
Global option to take precedence and/or be used as an upper limit.
To make this clearer, we can split out the options into Global and Repo
variants.
* test(util/exec): cover `Executing command` log line
As an important log line, we should make sure it doesn't change without
a test failing.
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
* chore(util/exec): log the environment variable names when exerting a command
When debugging, it's often unclear as to whether an environment variable
was passed to a command, unless we see the command fail (which includes
the environment variable names).
We can make sure we log the environment variable names themselves, when
executing the command, to make it more clear.
* fixup! chore(util/exec): log the environment variable names when exerting a command
---------
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
* fix(config/global): return default values for `GlobalConfig.get`
As noted in #39949, it can be awkward working with in `GlobalConfig.get`, where
we would previously need to specify the default value when calling `.get`,
which could lead to drift with the defaults for a given config option.
To reduce this risk, we can rely on a given `default` for a configuration
option, providing that as a default.
Because this now definitely returns a value that matches the config option, it
now allows us to remove `!` assertions.
* fixup! fix(config/global): return default values for `GlobalConfig.get`
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* fixup! fix(config/global): return default values for `GlobalConfig.get`
* test(presets): remove impossible test
As it's now not possible to set a value as `undefined` (as
`GlobalConfig` will then use the default), this test no longer validates
anything.
* test(workers/repository): add mock
Now `platform` defaults to `github`, we need to mock out the call to
fetch config.
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
* test: coverage
* test: coverage
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
* feat(managers/npm): allow specifying `toolSettings.nodeMaxMemory`
As noted in #40942, we're seeing an increase in memory usage across
`pnpm` and `yarn` users in Mend-hosted apps.
To allow limiting the maximum memory that any `node` child processes
will use, we can add a new `toolSettings.nodeMaxMemory` to (hopefully)
mitigate this.
Similar to the `toolSettings.jvmMaxMemory`, the self-hosted
administrator can set a maximum, which cannot be exceeded by repo-level
configuration.
* fixup! feat(managers/npm): allow specifying `toolSettings.nodeMaxMemory`
This change introduces a heap size limitation for the Gradle daemon used
when updating the Gradle wrapper, defaults to 512m.
The heap size is limited to the globally configured limit, which
defaults to 512m. It can be configured on the repository level, but not
exceed the globally configured limit. A hard coded mimiumum heap size of
512m is enforced.
The configuration option is named `toolSettings`, which has two
properties for the maximum and initial heap size. See
`docs/usage/configuration-options#toolSettings` for more details.
As part of future changes which will fix issues with Composer + Yarn 1
introduced by f430552de4, we need to find
a way to ignore command failure, on a per-command basis.
To do this, we can provide a refactor to introduce a new type,
`CommandWithOptions` which has the ability to specify whether to
`ignoreFailure`.
To make this less of a refactor than would otherwise require, we can
pass around a `(string | CommandWithOptions)[]`
(or a `string[]` or a `CommandWithOptions[]`) to be used, and use
helpers `asRawCommand` and `asRawCommands` to extract the underlying
command where necessary.
In the case a command's failure is being ignored, return the error code
in the `ExecResult`.
We also need to make sure that when executing in Docker, we still
provide the `|| true`s, otherwise we'll start to see commands failing
unexpectedly.
As part of future-proofing the `CommandWithOptions`, we can make sure
that it uses `string[]`s for the command it's representing, to avoid
re-parsing it.
As noted in #40242, without `shell` being passed around, this won't
correctly apply for Post Upgrade Tasks, and might break other usages.
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* fix(util/exec): migrate to `execa` for command execution
As part of a future change, we need to make sure that we have
cross-platform support for specific command executions.
By using `execa`'s functionality on top of `node:child_process`, we can
provide this functionality, as well as retain a consistent API with
internal users.
We need to make a couple of small tweaks to setting the `RawExecOptions`
as we're now bound by `readonly` properties.
As the `stdio` property's type is not exported, we can replace this with
an explicit setting of `stdin`, `stdout` and `stderr`.
This requires a number of test changes to make sure that the `stdin`,
`stdout` and `stderr` are set.
This doesn't migrate any calls under `tools`, as they can remain using
`node:child_process`.
* chore(util/exec): ignore floating promises `.on` calls
Since moving to use `execa`, we now have "floating promises" for the
`.on` method calls.
As we aren't handling them here, we can now mark them as `void` (as an
alternative to ignoring the ESLint rule).
This has been deprecated since 2022-07, and is unused as we only ever
use UTF-8.
In `tools/utils/exec.ts` we make sure to specify it explicitly, so we
don't get a Buffer returned.
* docs(constraints): clarify that `binarySource=global` ignores them
Although this is a somewhat commonly known behaviour, it's worth making
this more clear, by clarifying this in the `constraints` docs, as well
as add a debug log.
* docs(constraints): simplify message
As noted by Michael, these may not be user-facing `constraints`, but the
constraints a Manager sets.
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
---------
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
Jamie Tanna