fix(rubygems): preserve release timestamps for native gems
Include non-ruby platform suffixes when hashing release metadata so the identifiers match the RubyGems compact index. This prevents valid metadata from falling back to versions-only results and blocking minimum-release-age checks.
Cover generic, native, and absent platforms, cache reuse, mismatched platforms, and mixed-platform package lookups.
* refactor(datasource): stop double-caching releases at the index level
15 datasources set `caching = true`, which makes the registry wrapper in
`lib/modules/datasource/index.ts` store every `getReleases` result under
`datasource-releases-<id>` for 15 minutes, while their own `cached()`
wrapper already stores the same result under `datasource-<id>` with a
longer TTL and stale-fallback. Drop `caching` from those 15; the four
that rely on the index cache alone (dart-version, flutter-version,
maven, pypi) keep it.
`caching` is documented as requiring the datasource to return the
`isPrivate` flag, and none of these 15 ever set it, so the index layer
was caching private results unconditionally. The remaining `cached()`
layer honours `cachePrivatePackages` as before.
Also fix two cache-key/registry defects found alongside:
- `galaxy-collection` declares `registryStrategy = 'hunt'` but keyed its
cache on the package name only, so a `null` from the first hunted
registry was replayed for the second one. The registry URL is now part
of the key, which invalidates existing entries once.
- `typst` ignores `config.registryUrl` and always reads its default
registry, so it now sets `customRegistrySupport = false`. A configured
`registryUrls` for typst is now warned about and ignored instead of
being silently dropped.
* Merge branch 'main' into secustor/refactor/datasource-remove-index-caching
* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases
* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases
* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases
* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases
* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases
* add coverage
* fix
* fix
* fix
* code cov
* code cov
* code cov
* refactor(datasource): integrate GraalVM into java-version datasource
Merges graalvm-version datasource into java-version as a unified multi-vendor
Java distribution datasource. Implements vendor-based modular architecture with
routing layer for Adoptium (Eclipse Temurin) and Oracle GraalVM.
Architecture:
- Vendor-specific handlers (adoptium.ts, graalvm.ts) with routing in index.ts
- Vendor-aware system detection with different arch/os mappings per vendor
- Custom registry support for GraalVM's mise-java API
- Adoptium remains the default registry for backwards compatibility
Package names:
- New explicit: adoptium-jdk, adoptium-jre, oracle-graalvm-jdk, oracle-graalvm-jre
- Legacy (map to Adoptium): java, java-jdk, java-jre
Changes:
- Created adoptium.ts and graalvm.ts vendor handlers
- Enhanced common.ts with vendor detection and vendor-specific system mappings
- Updated index.ts with routing layer and custom registry support
- Merged test suites with comprehensive coverage for both vendors
- Updated mise manager integration to use JavaVersionDatasource
- Removed graalvm-version datasource directory
All 65 datasource tests and 93 mise manager tests passing.
* fix
* test(datasource): add coverage for non-HttpError paths in java-version
* fix(datasource): remove dead code and add coverage for java-version
Remove unreachable HttpError handling from _getReleases catch block
since both getAdoptiumReleases and getGraalvmReleases handle HttpError
internally. Add test for the graalvmRegistryUrl fallback when
registryUrl is undefined.
* fix(datasource): address review comments for GraalVM datasource
- Remove unused 'datasource-graalvm-version' cache namespace
- Convert MiseJavaRelease interface to Zod schema and derive type from it
- Use http.getJson() with the Zod schema in getGraalvmReleases instead
of getJsonUnchecked, letting the schema handle invalid/empty responses
- Update getGraalvmReleases test mock to use getJson
* docs: remove unused datasource-graalvm-version from cacheTtlOverride docs
* fix(datasource): preserve Adoptium config typing
* fix(datasource): harden GraalVM release parsing
* fix(datasource): address GraalVM review feedback
* test: cover GraalVM mise package mapping
* test(datasource): cover default GraalVM registry fallback
* test(datasource): cover legacy registry resolution
* test(datasource): type legacy datasource mocks
* fix(datasource): harden GraalVM defaults and responses
* refactor(datasource): use function-only registry accessors
* test(datasource): cover registry accessor functions
* test: restore coverage exclusions
* chore: retrigger CI checks
Every datasource that caches its releases hand-wrote the same
`withCache({ namespace: `datasource-${id}`, ... }, () => this._getReleases(config))`
wrapper, repeating the namespace literal and using underscore-prefixed
method names that `docs/development/best-practices.md` discourages.
Add `Datasource.cacheNamespace` (derived from the datasource id) and a
protected `Datasource.cached()` helper that defaults `namespace` to it,
exporting `CachedOptions` from `with-cache.ts`. Migrate 29 datasources
onto the helper, dropping their namespace literals, the static
`cacheNamespace` field in `github-digest`, and renaming the wrapped
`_getX` methods to `fetchX`.
Behaviour change: `golang-version` used the constant cache key `'all'`
while declaring `customRegistrySupport = true`, so a custom registryUrl
shared the cache entry with the default one. The key is now
`${config.registryUrl}`, which invalidates existing entries once.
Seven datasources (bitbucket-tags, bitbucket-server-tags,
gitlab-releases, gitea-tags, gitea-releases, forgejo-tags,
forgejo-releases) declared DatasourceApi metadata fields
(defaultRegistryUrls, releaseTimestampSupport/Note,
sourceUrlSupport/Note) as class `static readonly` instead of
instance fields. The contract (DatasourceApi, datasource.ts) reads
these off the instance, so the base class defaults silently applied
instead. Convert them to instance fields (`override readonly`, the
terraform-module pattern), keeping a static array only where a
static helper (or lib/modules/datasource/go/) still needs it, backed
by `override readonly defaultRegistryUrls = <Class>.defaultRegistryUrls`.
Delete the redundant `static readonly registryStrategy = 'first'`
(gitlab-releases, bitbucket-tags) and `static readonly
customRegistrySupport = true` (bitbucket-tags), which only matched
the base default by coincidence.
Replace the dead 'validates datasource' test in
lib/modules/datasource/index.spec.ts, which filtered out every
class-based datasource before validating (all 82 are class-based), with
a metadata contract test over getDatasources(): no listed metadata
name may be declared as a static-only property, releaseTimestampNote
requires releaseTimestampSupport, and sourceUrlNote requires
sourceUrlSupport !== 'none'.
Behaviour changes:
- These 7 datasources now report a real defaultRegistryUrls (or
releaseTimestampSupport/sourceUrlSupport) to the registry
(lib/modules/datasource/index.ts), so requests without an explicit
registryUrl now go through the registry-URL resolution and
getRegistryReleases package-cache path instead of calling
getReleases directly. getDigestConfig now receives a resolved
registryUrl instead of undefined for these datasources.
- Generated docs (tools/docs/datasources.ts,
tools/docs/minimum-release-age.ts) will render the real
"Release timestamp support" / "Source URL support" values and list
these datasources under minimumReleaseAge support, instead of "No".
* refactor(datasource): share the Gitea datasources with Forgejo
forgejo-tags and forgejo-releases were copies of gitea-tags and gitea-releases, and both -releases datasources re-implemented the digest lookup of the -tags ones. A GiteaDatasource base class now holds the shared implementation; the Forgejo datasources only declare their id, default registry URL, cache namespace and Http client. Results are only written to the shared package cache when they come from the public default instance.
* test(datasource): cover the default registry fallback
`GiteaDatasource.getRegistryUrl()` falls back to the instance's default
registry when the caller passes none, which the datasource dispatcher never
does because it resolves a registry URL first. Call `getTagCommit()` on the
instance instead, where the parameter is declared optional, once per default
registry so the Forgejo case also proves the base class reads the subclass's
`defaultRegistryUrls` rather than the Gitea static field.
* fix(datasource/crate): only query the crates.io web API for crates.io
Private sparse registries such as Artifactory or CodeArtifact advertise an
`api` URL in their `config.json` for publishing, but do not implement the
crates.io-only read endpoints `api/v1/crates/<name>` and
`api/v1/crates/<name>/<version>`. Since #41536 Renovate called both for
every registry with an `api` field, producing 404s and a
"Release interceptor failed" warning on each run.
Restore the crates.io flavor guard for the crate metadata lookup and the
release timestamp lookup. Reading `config.json` from a cloned registry is
now unreachable, so drop that branch.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(datasource/crate): cite the registry web API specification in the guards
* fix(datasource/crate): probe the registry web API once instead of guarding on flavor
Keep the crates.io read endpoints for registries that implement them.
Registries whose config.json points `api` at crates.io, such as index
mirrors, are used without probing. Any other registry is probed on the
first crate; a 404 for a crate that the index just returned means the
endpoint does not exist, so the registry is remembered as unsupported in
the memory cache and, for a day, in the package cache. The release
timestamp lookup handles a 404 the same way instead of surfacing the
generic "Release interceptor failed" warning.
Restores the cloned config.json branch, since cloned registries can now
advertise an API again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(datasource/crate): key the unsupported web API marker by the api URL
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
If we don't discover the image family for a given image, instead of
suggesting a package update that may not exist - as we may look up a
`defaultRegistryUrls` entry that doesn't match the distro we're using -
we should not suggest any update.
In these cases, we can specify the `skipReason: unknown-registry`.
In #45706, we'll try and auto-detect this where possible.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
feat(host-rules): ignore URL credentials when matching matchHost
`matchesHost()` compared the raw href of the searched URL against
`matchHost`, so a URL carrying `user:pass@` userinfo (often a literal
placeholder such as `${USER}:${PASS}@`) never matched a `matchHost`
without credentials, including path-scoped ones. Callers worked around
this by stripping `username`/`password` off the URL by hand before
calling `hostRules.find()`.
- Add `stripUrlCredentials()` to `lib/util/url.ts`, returning the URL's
`href` without userinfo (or `null` if unparseable), covered by
`lib/util/url.spec.ts`.
- `matchesHost()` in `lib/util/host-rules.ts` now strips credentials
from the searched URL before comparing it against an HTTP(S)
`matchHost`, so `hostRules.find()` ignores userinfo on every caller's
URL. Covered by new cases in `lib/util/host-rules.spec.ts` (a
`matchHost` rule matches a URL carrying `user:pass@`, including a
path-scoped `matchHost`).
- Remove the hand-rolled strip in
`lib/modules/datasource/pypi/host-rules.ts`'s
`findPypiIndexCredentials()`, now redundant since `find()` does it.
`lib/modules/datasource/pypi/host-rules.spec.ts` already covers this
behaviour and needed no changes.
This is strictly additive: more URLs match `matchHost` than before,
and nothing that matched before stops matching.
* refactor(python): share PyPI index credential resolution
pipenv, poetry, uv, pip-compile and the pypi datasource each resolved
index credentials differently, so a host rule that worked for one Python
manager did not necessarily work for the next. Add
`findPypiIndexCredentials()` in `lib/modules/datasource/pypi/host-rules.ts`,
which strips the URL's own `user:password@` part, runs a `pypi`-scoped
`hostRules.find()`, and only falls back to Google Artifact Registry when
the matched rule has no username and no password. The repeated
`.pkg.dev` test moves into a single `isGoogleArtifactRegistry()`
predicate next to `getGoogleAuthHostRule()`.
Behaviour changes:
- pipenv now supports Google Artifact Registry; previously it only ever
consulted host rules.
Spec: manager/pipenv/artifacts > "passes Google Artifact Registry
credential environment vars"
- pip-compile now supports Google Artifact Registry.
Spec: manager/pip-compile/common > getRegistryCredVarsFromPackageFiles()
> "supports Google Artifact Registry"
- pip-compile now matches `hostType: pypi` rules; its find call was
unscoped, so such rules were skipped.
Spec: manager/pip-compile/common > getRegistryCredVarsFromPackageFiles()
> "ignores a rule scoped to another hostType"
- pip-compile now matches a `matchHost` that contains a path; it reduced
every registry URL to its origin first, and `matchesHost()` compares
with `href.startsWith()`, so a path-scoped rule could never match.
Spec: manager/pip-compile/common > getRegistryCredVarsFromPackageFiles()
> "matches a rule scoped to a path"
- poetry and uv now strip `${USER}:${PASS}@` style placeholders from the
index URL before matching, so a `matchHost` can match those URLs too.
Spec: datasource/pypi/host-rules > "strips credentials from the URL
before matching"
- poetry now falls back to Google Artifact Registry when the matched rule
carries no username or password; any non-empty rule used to suppress
the fallback. Its dead unscoped `find({ url })` fallback is gone: a
`pypi`-scoped find already matches every rule an unscoped one would.
Spec: manager/poetry/artifacts > "falls back to Google auth when the
matching rule has no credentials"
- the pypi datasource now prefers host rule credentials over the Google
token for `*.pkg.dev`; it took the Google token unconditionally, and
the resulting `authorization` header made `applyHostRule()` skip the
configured credentials.
Spec: datasource/pypi/index > getReleases > "prefers host rule
credentials over Google Auth"
`getRegistryCredEnvVars()` also no longer logs the whole matched host
rule as the log merge object.
* test(pipenv): inline the Google Artifact Registry Pipfile
The fixture was used by a single spec case; inline it as requested in the
review of #46127.
* test(datasource/pypi): cover partial Google Artifact Registry credentials
A host rule that carries only a username or only a password still produces a
Basic header, but nothing exercised those two `?? ''` fallbacks, so the merge
queue failed the global branch threshold.
* feat(manager/dockerfile): extract `apt install` packages from `RUN` instructions
Similar to the support we've recently added for extracting `apk`
packages in `Dockerfile`s, we can also extract `apt install` and
`apt-get install`s.
We can build on top of our shared extraction layer, and:
- preserving an `:arch` qualifier when replacing the version
- skipping versions/constraints we can't update
Co-Authored-By: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* docs(manager/dockerfile): clarify where to find skipped logs
---------
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Now we have support for extracting `apk add`s in the Dockerfile manager,
we should also add a more in-depth indication of how to use this with
Wolfi or Chainguard Images.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* feat(manager/dockerfile): extract `apk add` packages from `RUN` instructions
As noted in #43711, users pin APK packages in their Dockerfiles and want
Renovate to update them, which so far has needed a custom manager[0].
Now that we have the `apk` datasource, we can extract these ourselves,
and parse the `RUN` instruction more effectively than a regex can.
We can do this by:
- extracting `apk add` dependencies when processing a `Dockerfile`
- ignoring common flags to `apk` before we extract the dependency names,
which wasn't straightforward with a Custom Manager
- ignoring i.e. `--virtual .build-deps`
- handle example cases from [0] and other usages i.e. with a comment in
between lines
- ensure that `final` doesn't run against the final `apk` install
- removing the docs for how to do this with a Custom Manager, as that
complexity is no longer needed
- skipping `unspecified-version` and `contains-variable`
We inform the user that they will need to override the `registryUrls`,
as we already have a default on the `apk` datasource and can't override
it in a straightforward means.
[0]: https://www.jvt.me/posts/2025/03/24/renovate-regex-dockerfile-apk/
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! feat(manager/dockerfile): extract `apk add` packages from `RUN` instructions
Co-Authored-By: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! feat(manager/dockerfile): extract `apk add` packages from `RUN` instructions
Co-Authored-By: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! feat(manager/dockerfile): extract `apk add` packages from `RUN` instructions
Find the command by name wherever it sits in the tokens, rather than
skipping an allowlist of shell keywords and variable assignments, so that
a wrapper program like `sudo`, `chroot /mnt` or `timeout 30` is handled
too.
Co-Authored-By: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! feat(manager/dockerfile): extract `apk add` packages from `RUN` instructions
---------
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* test(versioning): cover four uncovered guards
- `cargo`: a converted range whose operators are already attached to
their versions, so `npm2cargo` leaves the elements alone
- `hashicorp`: a widen on a complex range ending in `>=`, which the
underlying semver implementation refuses and returns null for
- `poetry`: a version list holding an unparseable entry, for both
`getSatisfyingVersion` and `minSatisfyingVersion`, plus the same widen
refusal, which falls back to the current value
- `swift`: an open-ended range whose bound is not a valid semver
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(versioning): drop two guards that cannot be false
- `hermit`: `major` is the only mandatory group in the version regex, so
the `if (major)` around the first release segment always holds; the
sibling parser already pushes it unconditionally
- `rpm`: after `c1 === c2` is ruled out, one of `c1 > c2` and `c1 < c2`
must hold, so the second comparison becomes a ternary
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(versioning/gradle): cover three uncovered guards
- `getMajor` on a version whose leading token is not a number
- `getSatisfyingVersion` with a later match that is not greater than the
one already held
- `minSatisfyingVersion` with a later match that is not smaller than the
one already held
Two branches in `compare.ts` are still open and are looked at next.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(versioning/ivy): cover a later match that is not the greatest
`getSatisfyingVersion` kept every matching version it walked past, so the
guard that holds on to the one already found was never taken.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(versioning): drop three guards that cannot be false
- `gradle`: `tokenCmp` compares two tokens whose only types are `Number`
and `String`, so once both-numbers and both-strings are handled
exactly one side is a number; the chain becomes a ternary and the
unreachable `return 0` goes
- `ivy`: `tokenize` always yields at least one token for a non-empty
version, so the length check around the last one always holds
- `ivy`: `rangeToStr` only returns null for a null range, which the
`length === 1` check above it already rules out
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(versioning/nuget): cover a later match that is not the greatest
Both loops in `getSatisfyingVersion` walked version lists that only ever
climbed, so the guard that holds on to the candidate already found was
never taken -- once for a bracket range, once for a bare version acting
as a minimum.
Also pin down that a floating range needs a major component: `-*` alone
does not parse.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(versioning): drop three more guards that cannot be false
- `maven`: the interval picked for auto-extension is only picked when the
new value is at or past its upper bound, and that upper bound is the
lower one incremented, so the new value never sorts below the lower
bound
- `maven`: an interval never has both bounds open, because the parser
only nulls one side when the other is a version
- `nuget`: a floating range always carries a major component, so neither
the parser's NaN check nor the serializer's undefined check can fail
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(versioning/pep440): drop three unreachable range paths
`updateRangeValue` handles `!=`, a `.*` prefix, `~=`, `==` and `<=`
itself, `handleUpperBound` handles `<`, and `parseCurrentRange` rejects
`===`. Only `>` and `>=` can reach `handleLowerBound`, so its operator
check always holds and it never returns null. Making it total removes
the null check at the call site as well.
This does drop the `logger.error` that followed, which was guarded by a
comment saying it could not happen unless pep440 itself changed.
`isLessThanRange` classifies the same fixed operator set, so the last
check there is the only one left and its fallback was dead.
`divideCompatibleReleaseRange` only ever sees a `~=` version, and pep440
requires at least two release components after that operator, so there
is always a component to drop. A new test pins down that `~=7` is not a
valid range.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(versioning): cover three more uncovered guards
- `elm`: a widen where the new version sits below the lower bound, so
the upper bound is kept rather than extended
- `hex`: a widen on a complex range ending in `>=`, which the underlying
semver implementation refuses and returns null for
- `rust-release-channel`: a concrete current version paired with a
channel name as the new version, which falls through to the new
version unchanged
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(versioning): drop two more guards that cannot be false
- `composer`: `getNewValue` falls back to the new version whenever
nothing else matched, so it never returns null and the recursive call
in the or-range path always has a value to work with
- `docker`: a version's prerelease is always a string, so once an empty
one on either side is handled, the remaining pair is both non-empty
and unequal
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(config,datasource): cover two more uncovered guards
- `config/validation`: `hostRules[].allowInternal` nested inside a
`packageRules[]` entry, which reports a plain configuration error
rather than a security one, because nested host rules are never applied
- `datasource/docker`: a non-list manifest whose response carries no
`docker-content-digest` header, so the digest falls back to the hash of
the manifest body
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(config,datasource): unwind three branches v8 could not count
- `config/decrypt` and `datasource/docker/common` each picked a response
with an `await` inside a ternary. v8 gives the branch that follows such
a ternary a negative hit count, which the reporters read as uncovered
even though both paths run. Spelling the choice as an if/else makes the
count come out right; behaviour is unchanged.
- `datasource/index` and `datasource/npm/npmrc` each ended an exhaustive
chain with an `else if` whose implicit else v8 counts but nothing can
reach: `registryStrategy` is one of three literals, and `npmrc` starts
as an object and is only ever reassigned to one.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(workers/global): cover seven uncovered guards
The config parsers rewrite legacy boolean spellings of `dryRun`,
`requireConfig` and `platformCommit`, but nothing passed a value that was
already current, so the fall-through of each chain went untaken. Cover it
from both the CLI and the environment.
Also cover a datasource env var that names no auth field, one whose value
is undefined, a default config file that cannot be read, and the error
`isLimitReached` throws when asked for a concurrent limit without a
branch config. That last one had an ignore hint saying it should not
happen; the error message documents the contract, so it is now tested and
the hint is gone.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(workers/global): drop a guard that cannot be false
`getConfig` in the additional-config-file parser returns early unless
`RENOVATE_ADDITIONAL_CONFIG_FILE` is set, so by the time a parse error is
being classified the variable is always present.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(workers/repository): drop a dead changelog file lookup
The gitea, gitlab and forgejo changelog sources each declared `files` as
an empty array and then guarded the only assignment to it with
`if (!files.length)`, which is always true one line after the
declaration. Collapse it to the filter itself.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* ci: ratchet the branch coverage threshold
A full run on this branch reports 100% statements (49471/49471), 100%
functions (7381/7381), 100% lines (48961/48961) and 99.73% branches
(25690/25759), up from 99.53% before this branch's work. The threshold
moves to 99.71, keeping the same rounding-down margin as before.
Statements, functions and lines are already pinned at 100 and stay
there.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(config,datasource): say why the ternary was unwound
Both sites now carry a comment explaining that the if/else is deliberate:
v8 gives the branch following an `await` inside a ternary a negative hit
count, which the coverage reporters read as uncovered.
`apiCheckResponse` also gets an explicit `HttpResponse<unknown>` type,
which is what the two branches have in common and what
`isECRMaxResultsResponse` already expects.
Also fix a type error in the new config file spec: the `pathExists` spy
needs the same `as never` cast the surrounding tests use.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* refactor(versioning/pep440): use a template literal in handleLowerBound
The exclude-style return is the one line this branch rewrote in that
function, so it now builds its string with a template literal instead of
the `+` operator.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(workers/global): drop the cast from the limits throw test
Calling the overload with a non-null assertion on the config argument
says the same thing without retyping the whole function.
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
---------
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* fix(datasource/go): cache the publication time of each version
The Go proxy defines that the `.info` API call is for a specific release
version of a module, this file should never be modified after it's been
created.
However, because we previously had a 30m TTL on these entries, a
repository that runs infrequently can lead to re-fetching data that
doesn't ever update.
We can fix this by introducing a separate cache namespace for the
module's creation timestamp (`Time` in the `.info` API call).
We can make sure that:
- we always merge, not replace, the entries, as the `major < 2` filter
in `_getReleases` means a run may only see a subset of a module's
versions, and writing back only what we saw would evict the rest
- we re-read immediately before writing, as `foo` and `foo/v2` are
separate lookups which both walk the `v2` module, so one can overwrite
the other's timestamps
- the key is the proxy which served the data, minus any credentials,
since a private Artifactory `GOPROXY` is a different universe of
timestamps to `proxy.golang.org`
- only the timestamps we successfully fetched are stored, so a proxy
which serves no `Time` doesn't have that recorded for 100 days
- it uses its own namespace, so that a `cacheTtlOverride` on the release
cache doesn't drag the immutable data down with it
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! fix(datasource/go): cache the publication time of each version
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! fixup! fix(datasource/go): cache the publication time of each version
Co-Authored-By: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
---------
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
As part of follow-up changes we'll cache the publication times we fetch
from a Go proxy, but should have coverage of the current behaviour
first.
As the Go proxy defines that the `.info` API call is for a specific
release version of a module, this file should never be modified after
it's been created.
However, because we only have a 30m TTL on these entries, a repository
that runs infrequently can lead to re-fetching data that doesn't ever
update.
Before we fix this, we can document the existing behaviour, by showing
that an expired `datasource-go-proxy` cache leads to re-requesting
`.info` API calls.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Previously, we were caching private Go modules' information, even when
`cachePrivatePackages=false`.
This leads to the module path, versions, and possibly the `go`
directive being cached unintentionally.
Instead of doing so, we should make sure that we're intentionally
caching packages.
To do this, we can use a mix of the `GOPRIVATE`/`GONOPROXY` to determine
which packages shouldn't be fetched in the public Go proxy, given
they're private, following how the Go toolchain itself does this.
When interacting with the public Go proxy, we treat these packages as
public (as the public Go proxy has no means to authenticate) and so
anything outside of this should be treated as private.
When `GOPROXY` contains `direct`, we only treat a package as public only
when the public proxy is in the list too, the way Go's own default is
written. This is used to imply that `direct` is the fallback for
whatever the public proxy doesn't have.
When thre is no Go proxy set, `GOPROXY=direct` means everything comes
from source, which tells us nothing about who is allowed to read it.
This doesn't catch everything: with the default `GOPROXY`, a private
package the public proxy doesn't have falls through to `direct` and is
still cached by Renovate, unless `GOPRIVATE` says otherwise. We only
learn that no proxy served it after we've fetched it, which is after
`cacheable` is determined.
When using Renovate with a private Go proxy mirror, such as Artifactory,
this will result in no cached packages until `cachePrivatePackages:
true` is set.
This is a similar change to #45348.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* test(versioning/aws-machine-image): cover the empty version guard
`_parse()` returns null for a falsy version before it tries the AMI id
regex, and no test passed one. Checking that an empty string is not
valid closes the only branch the module was missing.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource/buildpacks-registry): cover a response without a homepage
`latest.homepage` is optional in the registry response, but every test
supplied one, so the release result was never built without it. A
buildpack whose latest entry carries no homepage now checks that the
releases still come back and no source url is derived.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover optional-field branches in four datasources
Each of these datasources builds its release result by attaching fields
only when the registry supplies them, and the "not supplied" path was
never exercised:
- cdnjs: a library with neither homepage nor repository
- cpan: a search result where no release is marked latest, so no tag
- dart: a latest release carrying no pubspec
- ruby-version: a release table row whose first column is not a version
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover missing-metadata branches in hex and pypi
Two more datasources only attached fields when the registry supplied
them, and the absent case was untested: a hex package with neither an
html_url nor a github link in its meta, and a pypi response carrying no
releases at all.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover galaxy and repology fallbacks, mark unreachable guards
Adds a galaxy role with empty github fields, so no source url is
derived, and a repology lookup where the api fallback after a 403 finds
no matching package.
Marks four guards that are not worth a test: cpan and dart both check a
variable that is only null when the request threw, and
`handleGenericErrors()` always rethrows; conan's registry url is always
supplied by the datasource layer; and repology's resolver either returns
a body or throws.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover pod CDN branches, mark the sbt listing guards
Adds two pod lookups: an empty CDN body, and a CDN index that lists no
matching pod.
Marks the guards whose tests would need multi-level directory listing
fixtures: both sbt datasources walk nested index pages, and pod's github
helper checks a body that an empty json response never produces because
parsing throws first.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover maven metadata without a latest tag
Adds maven metadata carrying a release but no latest, so respectLatest
is left alone. Marks the three deb guards, which would each need a
crafted package index: a block missing a required key, a repeated
package in the final block, and two component indexes whose meta agrees.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover npmrc registry fallback, mark cache-bound guards
Adds a resolveRegistryUrl lookup for a package no npmrc rule matches.
Marks the guards whose tests would need a primed cache or a specific
blob shape: nuget's services index, deno's release cache, the docker hub
cache window, docker's manifest and config handling, go's pseudo-version
fallback, and both rubygems cache layers.
Three implicit elses are left counted: a coverage-ignore hint does not
attach to an `else if`, so the registry strategy chain, the npmrc reset
and docker's digest-header branch carry a plain note instead.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* fix(datasource/repology): pass the mock body as a string
`ResponseMock.body` is typed as a string, and every other call in the
spec passes `'[]'`. The new api fallback test passed an array literal,
which tsc rejected.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
---------
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* added support for extracting integrity and tarball from npm datasource
* test(datasource/npm): keep integrity and tarball in packument shape assertions
The packument-trimming tests on main pinned a cached shape that dropped
`dist.tarball`, so they broke once the schema started retaining
`integrity` and `tarball`. Update both expectations to the new shape and
cover `integrity` alongside `tarball`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(workers/lookup): expect integrity and tarball on npm updates
`generateUpdate` copies `downloadUrl` and `newDigest` from the release
onto the update, so every npm lookup in these tests now carries the
tarball URL and, where the packument fixture has one, the integrity
hash. Pin and rollback updates are built without those release fields,
so their expectations are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Michael Kriese <michael.kriese@mend.io>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test: close remaining line coverage gaps
A full coverage run reported three uncovered lines:
- `rez` versioning: `rez2pep440()` was never called with a plain version,
because `getNewValue()` short-circuits via `matchVersionPrecision()`
whenever the new version is valid for npm. A four-component rez version
is not, so it reaches the pep440 path.
- `nuget` v3: only the inner nuspec 404 was covered, never a 404 escaping
from the `.nupkg` download into the outer catch.
- `github` platform: the statement after `await` in the GitHub App branch
is a v8/source-map false negative - it runs, but is counted zero. The
branch used to carry the `v8 ignore` comment, which moved away with the
`ghHostname` block in #44587.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* build: ratchet coverage thresholds
A full run now reports 100% statements (49776/49776), 100% lines
(49256/49256), 100% functions and 98.72% branches (25756/26089), so the
thresholds move up accordingly. Branches keep the same rounding-down
margin as before.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
---------
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* fix(datasource/gitlab-tags): replace getJsonUnchecked with validated getJson
The go datasource embeds GitlabTagsDatasource, so its test mocks are
updated to satisfy the stricter GitlabTag schema (commit.id/created_at).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: fixup tests
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
To harden Renovate from being able to access internal hosts, we can add
a level of enforcement on all outgoing requests.
We can split this into two types of traffic:
- where the response becomes configuration (i.e. `extends`)
- where the response does not become configuration (i.e.
`customDatasources`)
To harden this, we enforce on every outbound HTTP request:
- check the resolved URL before anything, including cache lookups, can
proceed
- catching IP-literal hosts in every spelling the WHATWG URL parser normalizes
- blocking known Cloud metadata IPs, too
- a `dnsLookup` wrapper validates every address a hostname resolves to,
on every new connection, which also defeats DNS rebinding
- a `beforeRedirect` hook re-runs the URL check on each redirect
target, with the new host getting its own grant lookup rather than
inheriting the original host's verdict
We default this enforcement to `warn`, to not break existing deployments
and instead give them visibility into required configuration before a
future major version (major+1) enables `block` by default.
We'll also remove allow in a future major version (major+2), to give
time for admins to migrate over.
When a request is blocked, this degrades like an unreachable registry,
so a Renovate run isn't aborted by a blocked request.
As part of this, we can also provide a worked example of the different
cases that do/do not need allowing via `hostRules`.
Requests through an admin-configured proxy resolve DNS at the proxy, so
`dnsLookup` never runs for them, so we can resolve the hostname locally
and validate every address before the request instead.
This doesn't catch everything, especially as the proxy may resolve the
name to a different result, and if we can't resolve the name - possible
in locked-down corporate environments - we fail open.
Child processes aren't currently able to route traffic through
Renovate's HTTP layer, so are unrestricted.
Co-authored-by: Claude Fable 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
As part of future changes, we're introducing a means to parse
Dockerfiles for any `apk add` commands, so we can propose updates to any
packages using the APK datasource.
Before we do this, we need to correct a couple of bugs in our
implementation of the APK versioning scheme, so we can handle i.e.
apk add curl=~8.12.1
When the `apk` CLI builds a version constraint, it `OR`s one flag per
operator character[0].
For instance, `~` sets both "fuzzy" and "equal".
These operator characters may be in any order and may repeat, so `~`,
`=~` and `~=` are all the same operator, and `>>` and `>` are the same.
To correct our parsing, we should take this into account, rather than
using known patterns.
When we perform a fuzzy match, this is performed over version _tokens_
not the version itself, which means that:
- `~1.6` matches `1.6`, `1.6.0_pre1`, `1.6.0`, `1.6.5` and `1.6.9_p1`
- `~1.6` does not match `1.60`
- `~8.12.1` matches every `8.12.1-rN` i.e. `8.12.1-r1` and `8.12.1-r100`
While doing this, we also fix related issues:
- `~` previously treated this as an npm-style tilde ("same major and
minor, and greater than or equal"),
- admitted fuzzy versions that `apk` rejects - `~2.39.0-r0` should not have
matched `2.39.1-r0`
- rejected fuzzy versions that `apk` accepts - `~2.39.0` should have
matched `2.39.0-r1`
- `=~` and `~=` were not allowed
- `isSingleVersion('=~1.2.3')` was true, so a range looked like a pin
- `isLessThanRange` compared against the raw range, so `_parse` failed on
the operator and it always returned false
- `minSatisfyingVersion` was inherited, so it only ever found an exact
match
- `getNewValue` dropped the operator entirely, rewriting `~8.12.1` to a
bare version
We also set `supportsRanges`, which we'd forgotten to set despite `~`
and `>=` being ranges, and note that we support `pin`, `replace` and
`bump` strategies.
When using `getNewValue`, we now keep the precision the constraint was
written with, as otherwise we'd quietly narrow what the user asked for:
- `~8.12.1` becomes `~8.13.0` (previously `~8.13.0-r0`)
- `~8.12` becomes `~8.13`
- `~8.12.1-r0` becomes `~8.13.0-r2` (with a new revision number)
For `bump`, we raise a lower bound which includes its boundary, as with
`npm` and `pep440`, so:
- `>=8.12.1` becomes `>=8.13.0`
- `>~8.12` becomes `>~8.13`
Otherwise we return `null`, logging which constraint we couldn't handle
in the `bump` case, as `npm` does:
- `>` on its own can't be bumped, as `>8.13.0` would exclude the version
we bump to
- `<` and `<=` are upper bounds, so there's no lower bound to raise
- for `replace`, there's no single obvious new bound for `<`, `<=`, `>`,
`>=`, `>~` or `<~`, and a bound which already admits the new version
doesn't need one
- a prefix constraint whose version doesn't parse, i.e. `~foo`, which
would otherwise throw
`><` constrains a package to an identity hash rather than to a version,
so it's rejected outright.
Additionally, `isLessThanRange` and `matches` weren't agreeing on how to
resolve a fuzzy constraint.
For instance, `_compare` ranks `2.39.0_git20240101` below `2.39.0`, but
`~2.39.0` matches it, this would lead to a rollback of sorts.
Instead, we can ignore the `_git` and `_p` suffixes, which are common in
Alpine and Wolfi packages.
[0]: https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/44dcdfc255c26a4e19a4a27f66a54169cce5ca72/src/version.c
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* support AWS S3 as a chart repository
* support creds via hostRules
* apply suggestions
* fix(datasource/helm): classify s3 lookup errors
* test: use await expect().resolves
* fix(datasource/aws-machine-image): return all AMI releases for minimumReleaseAge filtering
Previously, _getReleases only returned the latest image in the releases array,
which broke minimumReleaseAge filtering. When the latest AMI was too new,
there were no older candidates to fall back to.
Fixed by:
- _getReleases now returns all matching AMIs as separate releases (sorted oldest→newest)
- _getDigest simplified to read directly from the images array to ensure latest digest is still returned
- Added reproducer test verifying all 3 images are returned
This aligns with how other datasources (npm, PyPI, etc.) work — they return
complete version history to allow minimumReleaseAge filtering to find an
appropriately-aged candidate.
Co-authored-by: Claude Code <claude@anthropic.com>
* chore: apply suggestion
* fixup! chore: apply suggestion
---------
Co-authored-by: Claude Code <claude@anthropic.com>
When we introduced the `apk` datasource, we missed that the default for
the Datasource was the `loose` versioning, which doesn't make as much
sense as `apk`.
This may be a small breaking change for users who are working with the
APK datasource already, but this is a minimal enough change that this is
fine, and can be restored with a Package Rule.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Jamie Tanna