Previously, we were caching private Go modules' information, even when
`cachePrivatePackages=false`.
This leads to the module path, versions, and possibly the `go`
directive being cached unintentionally.
Instead of doing so, we should make sure that we're intentionally
caching packages.
To do this, we can use a mix of the `GOPRIVATE`/`GONOPROXY` to determine
which packages shouldn't be fetched in the public Go proxy, given
they're private, following how the Go toolchain itself does this.
When interacting with the public Go proxy, we treat these packages as
public (as the public Go proxy has no means to authenticate) and so
anything outside of this should be treated as private.
When `GOPROXY` contains `direct`, we only treat a package as public only
when the public proxy is in the list too, the way Go's own default is
written. This is used to imply that `direct` is the fallback for
whatever the public proxy doesn't have.
When thre is no Go proxy set, `GOPROXY=direct` means everything comes
from source, which tells us nothing about who is allowed to read it.
This doesn't catch everything: with the default `GOPROXY`, a private
package the public proxy doesn't have falls through to `direct` and is
still cached by Renovate, unless `GOPRIVATE` says otherwise. We only
learn that no proxy served it after we've fetched it, which is after
`cacheable` is determined.
When using Renovate with a private Go proxy mirror, such as Artifactory,
this will result in no cached packages until `cachePrivatePackages:
true` is set.
This is a similar change to #45348.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* test(versioning/aws-machine-image): cover the empty version guard
`_parse()` returns null for a falsy version before it tries the AMI id
regex, and no test passed one. Checking that an empty string is not
valid closes the only branch the module was missing.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource/buildpacks-registry): cover a response without a homepage
`latest.homepage` is optional in the registry response, but every test
supplied one, so the release result was never built without it. A
buildpack whose latest entry carries no homepage now checks that the
releases still come back and no source url is derived.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover optional-field branches in four datasources
Each of these datasources builds its release result by attaching fields
only when the registry supplies them, and the "not supplied" path was
never exercised:
- cdnjs: a library with neither homepage nor repository
- cpan: a search result where no release is marked latest, so no tag
- dart: a latest release carrying no pubspec
- ruby-version: a release table row whose first column is not a version
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover missing-metadata branches in hex and pypi
Two more datasources only attached fields when the registry supplied
them, and the absent case was untested: a hex package with neither an
html_url nor a github link in its meta, and a pypi response carrying no
releases at all.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover galaxy and repology fallbacks, mark unreachable guards
Adds a galaxy role with empty github fields, so no source url is
derived, and a repology lookup where the api fallback after a 403 finds
no matching package.
Marks four guards that are not worth a test: cpan and dart both check a
variable that is only null when the request threw, and
`handleGenericErrors()` always rethrows; conan's registry url is always
supplied by the datasource layer; and repology's resolver either returns
a body or throws.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover pod CDN branches, mark the sbt listing guards
Adds two pod lookups: an empty CDN body, and a CDN index that lists no
matching pod.
Marks the guards whose tests would need multi-level directory listing
fixtures: both sbt datasources walk nested index pages, and pod's github
helper checks a body that an empty json response never produces because
parsing throws first.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover maven metadata without a latest tag
Adds maven metadata carrying a release but no latest, so respectLatest
is left alone. Marks the three deb guards, which would each need a
crafted package index: a block missing a required key, a repeated
package in the final block, and two component indexes whose meta agrees.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* test(datasource): cover npmrc registry fallback, mark cache-bound guards
Adds a resolveRegistryUrl lookup for a package no npmrc rule matches.
Marks the guards whose tests would need a primed cache or a specific
blob shape: nuget's services index, deno's release cache, the docker hub
cache window, docker's manifest and config handling, go's pseudo-version
fallback, and both rubygems cache layers.
Three implicit elses are left counted: a coverage-ignore hint does not
attach to an `else if`, so the registry strategy chain, the npmrc reset
and docker's digest-header branch carry a plain note instead.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* fix(datasource/repology): pass the mock body as a string
`ResponseMock.body` is typed as a string, and every other call in the
spec passes `'[]'`. The new api fallback test passed an array literal,
which tsc rejected.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
---------
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* added support for extracting integrity and tarball from npm datasource
* test(datasource/npm): keep integrity and tarball in packument shape assertions
The packument-trimming tests on main pinned a cached shape that dropped
`dist.tarball`, so they broke once the schema started retaining
`integrity` and `tarball`. Update both expectations to the new shape and
cover `integrity` alongside `tarball`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(workers/lookup): expect integrity and tarball on npm updates
`generateUpdate` copies `downloadUrl` and `newDigest` from the release
onto the update, so every npm lookup in these tests now carries the
tarball URL and, where the packument fixture has one, the integrity
hash. Pin and rollback updates are built without those release fields,
so their expectations are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Michael Kriese <michael.kriese@mend.io>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test: close remaining line coverage gaps
A full coverage run reported three uncovered lines:
- `rez` versioning: `rez2pep440()` was never called with a plain version,
because `getNewValue()` short-circuits via `matchVersionPrecision()`
whenever the new version is valid for npm. A four-component rez version
is not, so it reaches the pep440 path.
- `nuget` v3: only the inner nuspec 404 was covered, never a 404 escaping
from the `.nupkg` download into the outer catch.
- `github` platform: the statement after `await` in the GitHub App branch
is a v8/source-map false negative - it runs, but is counted zero. The
branch used to carry the `v8 ignore` comment, which moved away with the
`ghHostname` block in #44587.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* build: ratchet coverage thresholds
A full run now reports 100% statements (49776/49776), 100% lines
(49256/49256), 100% functions and 98.72% branches (25756/26089), so the
thresholds move up accordingly. Branches keep the same rounding-down
margin as before.
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
---------
Co-authored-by: Claude Opus 5 <michael.kriese+claude-code@mend.io>
* fix(datasource/gitlab-tags): replace getJsonUnchecked with validated getJson
The go datasource embeds GitlabTagsDatasource, so its test mocks are
updated to satisfy the stricter GitlabTag schema (commit.id/created_at).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: fixup tests
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
To harden Renovate from being able to access internal hosts, we can add
a level of enforcement on all outgoing requests.
We can split this into two types of traffic:
- where the response becomes configuration (i.e. `extends`)
- where the response does not become configuration (i.e.
`customDatasources`)
To harden this, we enforce on every outbound HTTP request:
- check the resolved URL before anything, including cache lookups, can
proceed
- catching IP-literal hosts in every spelling the WHATWG URL parser normalizes
- blocking known Cloud metadata IPs, too
- a `dnsLookup` wrapper validates every address a hostname resolves to,
on every new connection, which also defeats DNS rebinding
- a `beforeRedirect` hook re-runs the URL check on each redirect
target, with the new host getting its own grant lookup rather than
inheriting the original host's verdict
We default this enforcement to `warn`, to not break existing deployments
and instead give them visibility into required configuration before a
future major version (major+1) enables `block` by default.
We'll also remove allow in a future major version (major+2), to give
time for admins to migrate over.
When a request is blocked, this degrades like an unreachable registry,
so a Renovate run isn't aborted by a blocked request.
As part of this, we can also provide a worked example of the different
cases that do/do not need allowing via `hostRules`.
Requests through an admin-configured proxy resolve DNS at the proxy, so
`dnsLookup` never runs for them, so we can resolve the hostname locally
and validate every address before the request instead.
This doesn't catch everything, especially as the proxy may resolve the
name to a different result, and if we can't resolve the name - possible
in locked-down corporate environments - we fail open.
Child processes aren't currently able to route traffic through
Renovate's HTTP layer, so are unrestricted.
Co-authored-by: Claude Fable 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
As part of future changes, we're introducing a means to parse
Dockerfiles for any `apk add` commands, so we can propose updates to any
packages using the APK datasource.
Before we do this, we need to correct a couple of bugs in our
implementation of the APK versioning scheme, so we can handle i.e.
apk add curl=~8.12.1
When the `apk` CLI builds a version constraint, it `OR`s one flag per
operator character[0].
For instance, `~` sets both "fuzzy" and "equal".
These operator characters may be in any order and may repeat, so `~`,
`=~` and `~=` are all the same operator, and `>>` and `>` are the same.
To correct our parsing, we should take this into account, rather than
using known patterns.
When we perform a fuzzy match, this is performed over version _tokens_
not the version itself, which means that:
- `~1.6` matches `1.6`, `1.6.0_pre1`, `1.6.0`, `1.6.5` and `1.6.9_p1`
- `~1.6` does not match `1.60`
- `~8.12.1` matches every `8.12.1-rN` i.e. `8.12.1-r1` and `8.12.1-r100`
While doing this, we also fix related issues:
- `~` previously treated this as an npm-style tilde ("same major and
minor, and greater than or equal"),
- admitted fuzzy versions that `apk` rejects - `~2.39.0-r0` should not have
matched `2.39.1-r0`
- rejected fuzzy versions that `apk` accepts - `~2.39.0` should have
matched `2.39.0-r1`
- `=~` and `~=` were not allowed
- `isSingleVersion('=~1.2.3')` was true, so a range looked like a pin
- `isLessThanRange` compared against the raw range, so `_parse` failed on
the operator and it always returned false
- `minSatisfyingVersion` was inherited, so it only ever found an exact
match
- `getNewValue` dropped the operator entirely, rewriting `~8.12.1` to a
bare version
We also set `supportsRanges`, which we'd forgotten to set despite `~`
and `>=` being ranges, and note that we support `pin`, `replace` and
`bump` strategies.
When using `getNewValue`, we now keep the precision the constraint was
written with, as otherwise we'd quietly narrow what the user asked for:
- `~8.12.1` becomes `~8.13.0` (previously `~8.13.0-r0`)
- `~8.12` becomes `~8.13`
- `~8.12.1-r0` becomes `~8.13.0-r2` (with a new revision number)
For `bump`, we raise a lower bound which includes its boundary, as with
`npm` and `pep440`, so:
- `>=8.12.1` becomes `>=8.13.0`
- `>~8.12` becomes `>~8.13`
Otherwise we return `null`, logging which constraint we couldn't handle
in the `bump` case, as `npm` does:
- `>` on its own can't be bumped, as `>8.13.0` would exclude the version
we bump to
- `<` and `<=` are upper bounds, so there's no lower bound to raise
- for `replace`, there's no single obvious new bound for `<`, `<=`, `>`,
`>=`, `>~` or `<~`, and a bound which already admits the new version
doesn't need one
- a prefix constraint whose version doesn't parse, i.e. `~foo`, which
would otherwise throw
`><` constrains a package to an identity hash rather than to a version,
so it's rejected outright.
Additionally, `isLessThanRange` and `matches` weren't agreeing on how to
resolve a fuzzy constraint.
For instance, `_compare` ranks `2.39.0_git20240101` below `2.39.0`, but
`~2.39.0` matches it, this would lead to a rollback of sorts.
Instead, we can ignore the `_git` and `_p` suffixes, which are common in
Alpine and Wolfi packages.
[0]: https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/44dcdfc255c26a4e19a4a27f66a54169cce5ca72/src/version.c
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* support AWS S3 as a chart repository
* support creds via hostRules
* apply suggestions
* fix(datasource/helm): classify s3 lookup errors
* test: use await expect().resolves
* fix(datasource/aws-machine-image): return all AMI releases for minimumReleaseAge filtering
Previously, _getReleases only returned the latest image in the releases array,
which broke minimumReleaseAge filtering. When the latest AMI was too new,
there were no older candidates to fall back to.
Fixed by:
- _getReleases now returns all matching AMIs as separate releases (sorted oldest→newest)
- _getDigest simplified to read directly from the images array to ensure latest digest is still returned
- Added reproducer test verifying all 3 images are returned
This aligns with how other datasources (npm, PyPI, etc.) work — they return
complete version history to allow minimumReleaseAge filtering to find an
appropriately-aged candidate.
Co-authored-by: Claude Code <claude@anthropic.com>
* chore: apply suggestion
* fixup! chore: apply suggestion
---------
Co-authored-by: Claude Code <claude@anthropic.com>
When we introduced the `apk` datasource, we missed that the default for
the Datasource was the `loose` versioning, which doesn't make as much
sense as `apk`.
This may be a small breaking change for users who are working with the
APK datasource already, but this is a minimal enough change that this is
fine, and can be restored with a Package Rule.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fix(docker): adjust tag caching for private registries
Tag caching for private registries currently has two issues, observed in
issues #45249 and #40734:
1. `getTags` has no `cacheable` guard (unlike `getReleases`)
2. `getTags` caches `null` tag results
These two combined mean that regardless of `cachePrivatePackages`
settings, private registry tag lookups which fail due to transient/auth
errors are cached. This results in the cache being "poisoned" by
misconfigured jobs, with well-configured jobs unable to recover. This is
especially relevant for the hosted app where caching is shared by
different tenants.
This commit:
1. Aligns `getTags` caching with `getReleases`, ensuring caching
behaviour is driven by `cachePrivatePackages`.
2. Prevents `null` tag results being cached, allowing for recovery
regardless of `cachePrivatePackages` configuration.
Note that I have _not_ applied (2) to `getReleases`, because
1. `getReleases` has a slightly different contract in that it sets
`fallback: true` to intentionally cache failures
2. I have not observed any actual failures in `getReleases` due to (20,
so it hasn't been empirically proven as requiring a fix
3. `getTags` is internal so is a lower-risk change
Fixes
- #45249
Relates to
- #45249
- #40734
* refactor(docker): undefined for no-auth
Previously `getAuthHeaders()` used `null` as an auth-failed sentinel,
which was inconsistent with `withCache()`, and led to the original "fix"
where nulls were explicitly omitted from the cache.
Instead this implements the suggestion from https://github.com/renovatebot/renovate/pull/45348#issuecomment-5340524967
to uniformly return undefined for `getTags` auth errors, which allows
the cache to recognise failures itself. Then cacheability ends up being
conditional on dockerhub as always.
This also rectifies a mismatch between auth-failure signals, where other
functions like `_getImageConfig` were returning `undefined` when headers
were unavailable.
IMPORTANT: once this lands, since there aren't any cache invalidations,
the existing issue will still be observed until the cache TTL expires
and old `null` entries are evicted.
* fix: whoops, sort out the vacuous tests
I failed to realise caching isn't automatically orchestrated in these
tests. As an eagle-eyed observer [pointed out][1], this meant the tests
weren't actually exercising anything.
This commit fixes that by adding a new test block which does the cache
orchestration using a temp dir cache. I've also opted to add assertions
against the mock server's http trace counter directly to ensure the mock
really is/isn't being hit. This should also serve to make the tests fail
fast if the underlying impl flow (auth probe -> get tags) is adjusted in
the future.
I've also brought in some caching dependencies into the test file, but
those are used elsewhere too (e.g. `withCache` tests).
This time, I have actually verified the tests which we'd expect to fail
do fail against main. Note that the first 'caches tags for Docker Hub'
is expected to pass on both branches (the control test).
```
❯ getTags caching (3)
✓ caches tags for Docker Hub 21ms 249 MB heap used
× does not cache tags for a private registry 20ms 212 MB heap used
× does not cache failed lookups when cachePrivatePackages is enabled 15ms 215 MB heap used
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Tests 2 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
FAIL lib/modules/datasource/docker/index.spec.ts > modules/datasource/docker/index > getTags caching > does not cache tags for a private registry
AssertionError: expected [ '1.0.0' ] to deeply equal [ '1.0.0', '2.0.0' ]
- Expected
+ Received
[
"1.0.0",
- "2.0.0",
]
❯ lib/modules/datasource/docker/index.spec.ts:3694:72
3692| expect(httpMock.getTrace()).toHaveLength(2);
3693|
3694| expect(await ds.getTags('https://registry.company.com', 'node')).toEqual([
| ^
3695| '1.0.0',
3696| '2.0.0',
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/4]⎯
FAIL lib/modules/datasource/docker/index.spec.ts > modules/datasource/docker/index > getTags caching > does not cache tags for a private registry
Error: *** Unused HTTP mocks ***
- GET https://registry.company.com/v2/node/tags/list
Requests done:
- GET https://registry.company.com/v2/node/tags/list?n=10000 [200]
- GET https://registry.company.com/v2/node/tags/list?n=10000 [200]
---
Renovate testing strategy requires that every HTTP request
has a corresponding mock.
This error occurs because some of the created mocks are unused.
In most cases, you simply need to remove them.
+++
❯ <test> lib/modules/datasource/docker/index.spec.ts:3675:9
3673| });
3674|
3675| it('does not cache tags for a private registry', async () => {
| ^
3676| httpMock
3677| .scope('https://registry.company.com/v2')
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/4]⎯
FAIL lib/modules/datasource/docker/index.spec.ts > modules/datasource/docker/index > getTags caching > does not cache failed lookups when cachePrivatePackages is enabled
AssertionError: expected null to be undefined
- Expected:
undefined
+ Received:
null
❯ lib/modules/datasource/docker/index.spec.ts:3717:9
3715| expect(
3716| await ds.getTags('https://registry.company.com', 'node'),
3717| ).toBeUndefined();
| ^
3718| expect(httpMock.getTrace()).toHaveLength(1);
3719|
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/4]⎯
FAIL lib/modules/datasource/docker/index.spec.ts > modules/datasource/docker/index > getTags caching > does not cache failed lookups when cachePrivatePackages is enabled
Error: *** Unused HTTP mocks ***
- GET https://registry.company.com/v2/node/tags/list
Requests done:
- GET https://registry.company.com/v2/node/tags/list?n=10000 [403]
---
Renovate testing strategy requires that every HTTP request
has a corresponding mock.
This error occurs because some of the created mocks are unused.
In most cases, you simply need to remove them.
+++
❯ <test> lib/modules/datasource/docker/index.spec.ts:3701:9
3699| });
3700|
3701| it('does not cache failed lookups when cachePrivatePackages is enabled', async () => {
| ^
3702| GlobalConfig.set({ cachePrivatePackages: true });
3703|
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/4]⎯
Test Files 1 failed (1)
Tests 2 failed | 313 passed (315)
```
[1]: https://github.com/renovatebot/renovate/pull/45348#pullrequestreview-5092478059
* oxlint-fix
---------
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
As this slipped past our code review, but `matchPaths` has been
deprecated for ~3 years, replaced by `matchFileNames`.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* feat(datasource): add APK datasource support
* remove missing manager link for now
* lint docs
* update tests
* replace binary fixtures
* fix coverage
* Update lib/modules/datasource/apk/readme.md
Co-authored-by: Jamie Tanna <github@jamietanna.co.uk>
* Apply suggestion from @viceice
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
* use tar instead of tar-stream
* use codeBlock
* add ignore back
* make method private
* use an async extract instead of list
* add usage example
* stream apkindex download and parse line-by-line
* refactor(datasource): address review comments on APK datasource
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore: sync pnpm-lock.yaml after merge
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: remove ignored lines
As they seem to be covered, anyway.
* fixup! Merge remote-tracking branch 'origin/HEAD' into apk-datasource
* fixup! fixup! Merge remote-tracking branch 'origin/HEAD' into apk-datasource
* lint: use `node:` prefix
* Update lib/modules/datasource/apk/index.ts
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
* refactor(datasource/apk): use asTimestamp millis instead of new Date
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(datasource/apk): rename uncached method to _getPackages
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(datasource/apk): encode registry URL parts as query parameters
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(datasource/apk): reject invalid registry URLs, skip caching empty indexes
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(datasource/apk): throw on unparseable registry URL and drop unset sourceUrl support
Co-authored-by: Cursor <cursoragent@cursor.com>
* docs(datasource/apk): show how registry URL parameters build the index URLs
Co-authored-by: Cursor <cursoragent@cursor.com>
* Apply suggestions from code review
Co-authored-by: RahulGautamSingh <rahultesnik@gmail.com>
* fix(datasource/apk): keep credentials and port in component URLs
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(datasource/apk): extract APKINDEX by basename and key packages by name
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(datasource/apk): report a version served by two components only once
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Jamie Tanna <github@jamietanna.co.uk>
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
Co-authored-by: RahulGautamSingh <rahultesnik@gmail.com>
When a datasource uses the 'merge' registryStrategy (e.g. maven),
mergeRegistries() tags each release with the registryUrl of the repo
that actually reported it. postprocessRelease() ignored this and used
the first configured/default registryUrl instead, causing the Maven
datasource to probe the wrong repository for release verification.
If the artifact didn't exist in that first repo, the release was
wrongly rejected and no update was proposed, even though a valid
release existed in another configured repository.
* test: reduce oversized expectation literals to representative examples
* test: shrink oversized fixtures instead of trimming assertions
Cut the real-world fixtures down to a representative set of entries that still covers every parser shape, and assert the complete output again with a single toEqual.
* refactor(lint): enable oxlint prefer-named-capture-group at error
Numbered capture groups (m[1], $1) break silently when a regex is
edited and group order shifts. Named groups (?<name>...) make intent
explicit and accesses (m.groups.name) resilient to reordering.
This enables the built-in eslint rule prefer-named-capture-group,
surfacing 321 pre-existing unnamed-group violations across ~121
files to be burned down incrementally in follow-up commits.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(versioning): name capture groups for prefer-named-capture-group
Burns down all prefer-named-capture-group violations in
lib/modules/versioning/. Unused capture groups (only reachable via
.test() or whole-match) become non-capturing (?:...); groups whose
value is read become named (?<name>...), with accesses rewritten to
match.groups!.<name> (or destructured from .groups!) instead of
positional match[n]/$n so a future edit that reorders groups cannot
silently break the reader.
Two cases needed extra care to preserve behavior:
- semver/common.ts and poetry/patterns.ts split() on a regex whose
capture group is relied upon by String.prototype.split to keep the
separator in the output array; the group is kept capturing and
named, not converted to non-capturing.
- hashicorp/convertor.ts's semverRegex is only ever embedded via
.source into a named `(?<version>...)` group elsewhere and never
matched directly, so its internal groups are safe to make
non-capturing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(datasource): name capture groups for prefer-named-capture-group
Burns down all prefer-named-capture-group violations under
lib/modules/datasource/. Same treatment as the versioning module:
unused groups become non-capturing, used groups get names, and
positional match[n] reads move to match.groups!.<name>.
docker/ecr.ts and pypi/common.ts keep their positional destructuring
(`const [, region] = ...`, `!!m &&`) working unchanged since a named
capture group is still addressable by its numeric index in
JavaScript — only pypi/common.ts's access was rewritten to
m.groups!.owner for clarity since it read a single group directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(platform): name capture groups for prefer-named-capture-group
Burns down all prefer-named-capture-group violations under
lib/modules/platform/. Uses the repo's established idiom of
`match.groups?.<name>` after an optional-chained exec()/match() (see
bitbucket/index.ts hostnameWithoutApiPrefix, gerrit/utils.ts
extractSourceBranch) and `$<name>` in String.replace replacement
strings. Unused groups (markdown/HTML massaging regexes matched only
to be stripped) become non-capturing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(manager): name capture groups in ansible-galaxy for prefer-named-capture-group
Burns down prefer-named-capture-group violations in
lib/modules/manager/ansible-galaxy/. newBlockRegEx and blockLineRegEx
dropped their unused outer wrapping group (only the inner key/value
groups were ever read) and named the key/value groups; nameMatchRegex's
remaining unused groups (protocol/user-info/port alternatives) become
non-capturing since only source/hostname/depName/version are read via
.groups. Accesses in collections.ts and roles.ts move from
lineMatch[2]/lineMatch[3] to lineMatch.groups!.key/value.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(manager): name capture groups for prefer-named-capture-group (batch 1)
Burns down prefer-named-capture-group violations across the first
alphabetical slice of lib/modules/manager/ (ansible through mix).
Used groups get names with accesses rewritten to match.groups!.<name>
(or $<name> in replacement strings); groups whose value is never read
become non-capturing (?:...). Two String.split() call sites
(git-submodules/extract.ts, meteor/extract.ts) keep the established
"named group still readable positionally from the split() result
array" idiom instead of restructuring the destructuring.
bazel/rules/go.ts and buildpacks/extract.ts dockerRef verified
byte-for-byte equivalent before/after via a quick node script since
several nested groups collapsed to non-capturing at once.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(manager): name capture groups for prefer-named-capture-group (batch 2)
Burns down prefer-named-capture-group violations across the remaining
alphabetical slice of lib/modules/manager/ (npm through terragrunt).
Same treatment as batch 1: used groups get names with accesses moved
to match.groups!.<name> or $<name> replacement strings; unused groups
(wrapping alternations, quantifier repetition groups only consumed via
.test(), or values never read from a match) become non-capturing.
terraform/extractors/others/modules.ts and terragrunt/modules.ts share
near-identical URL-parsing regexes (github/bitbucket/git/azure-devops
ref matchers) — the same set of always-unused `depth=`/`&depth=`
query-string groups is converted to non-capturing in both files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(config): name capture groups for prefer-named-capture-group
Burns down prefer-named-capture-group violations in lib/config/.
Used groups get names with replacement strings switched from $1/$2 to
$<name>; the unused separator group in schedule-migration.ts's
afterBeforeRegex becomes non-capturing. presets/parse.ts keeps
assigning through the already-declared `repo` variable rather than
array-destructuring the exec() result, matching this file's existing
style for the other named-group regexes just above it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(util): name capture groups for prefer-named-capture-group
Burns down prefer-named-capture-group violations across lib/util/ and
lib/logger/. Same treatment as previous batches: named groups for
values that are read (via $<name> replacement strings, .groups!.<name>,
or destructured from .groups), non-capturing groups where nothing reads
the captured value (e.g. semantic.ts's angular-commit .test()-only
regex). www-authenticate.ts's tokenizer entries each get their own
named group but keep accessing the match generically via match[1],
since a positionally-indexed named group is still readable by index.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(workers): name capture groups for prefer-named-capture-group
Burns down prefer-named-capture-group violations in lib/workers/.
Used groups get names with $<name> replacement strings; unused
wrapper/alternation groups (e.g. semantic-commit-message.ts's optional
scope/issue wrappers, release-notes.ts's date-heading and
horizontal-rule matchers that are only probed with .search()/replaced
wholesale) become non-capturing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor(test,tools): name capture groups for prefer-named-capture-group
Finishes the prefer-named-capture-group burndown: test/, tools/, and
vitest.config.mts. Groups whose value is read get names ($<name>
replacement strings or .groups.<name> access); groups only used via
.test()/.search() or that feed a .match(/…/g) call (whose return value
is the array of whole matches, not captured groups) become
non-capturing.
This is the last remaining slice of pre-existing violations, so the
rule (already enabled at error repo-wide) now passes with zero
findings across the whole codebase.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* fix prettier failure
* refactor(lint): use a named capture group in `v8-ignore-no-count`
New rule from main trips the newly enabled `prefer-named-capture-group` oxlint rule.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(lint): add renovate/prefer-is-helpers and renovate/prefer-is-object rules
Flag ad-hoc type checks that have @sindresorhus/is equivalents:
- .filter(Boolean) -> .filter(isTruthy)
- typeof x === 'string' -> isString(x)
- x === null || x === undefined -> isNullOrUndefined(x) (and negation)
The typeof x === 'object' shape lives in a separate suggestion-only rule
(renovate/prefer-is-object, kept at warn) because isObject()/isPlainObject()
are not drop-in equivalents: null, arrays and functions behave differently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: fix renovate/prefer-is-helpers violations
Replace ad-hoc type checks with @sindresorhus/is helpers across lib/:
- 13x .filter(Boolean) -> .filter(isTruthy)
- 36x typeof x ==/!== 'string' -> isString(x) / !isString(x)
- 6x x === null || x === undefined -> isNullOrUndefined(x) (and negation)
Promote renovate/prefer-is-helpers to error now that the codebase is clean.
renovate/prefer-is-object stays at warn: its 15 remaining sites guard
parsed YAML/JSON or unknown values where isObject()/isPlainObject() are
not provably equivalent (null/array/function semantics).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: enforce renovate/prefer-is-object at error level
Resolve the 15 remaining warnings and raise the rule from warn to error:
- Migrate 10 sites to isObject() where the guarded value provably cannot
be a function (parsed YAML/JSON, typed exec command arguments, or
test-controlled fixtures): helm-values, woodpecker, bun, pre-commit
managers, exec utils, file package cache, and the github http spec.
In the bun manager this also fixes a latent crash where a null
`workspaces` value would throw on the `in` operator.
- Keep 5 sites as reasoned oxlint-disable comments where excluding
functions is load-bearing: the byte-exact fingerprint canonicalizer
(2), the circularity walker, the zod result guard, and the generic
hasKey helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(lint): add renovate/inline-single-use-fixtures rule
In spec files, a module-scope const fixture that is referenced from
exactly one it()/test() block should be inlined into that test so the
test is self-contained. The rule is a warn-level nudge (naming for
clarity is a valid exception) with no autofix (moving code is risky).
Uses oxlint jsPlugins scope analysis (getDeclaredVariables) for exact,
shadowing-safe reference counting and bails out conservatively: only
top-level non-exported single-declarator consts with multiline,
non-function initializers whose every reference sits inside the callback
function of one and the same test call are flagged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: inline single-use spec fixtures into their tests
Fix 26 renovate/inline-single-use-fixtures warnings in specs where the
move is mechanical and value-preserving: codeBlock fixtures were
re-indented uniformly (codeBlock strips the common indentation, so the
produced string is unchanged) and Fixtures.get() calls resolve relative
to the spec file, so relocating them inside the test body has no effect.
The remaining 59 warnings are left for per-site judgment, as the rule is
a warn-level nudge by design.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: inline remaining relocation-safe single-use spec fixtures
Move the 24 remaining single-use module-scope fixtures whose initializer
is a codeBlock tagged template or a Fixtures accessor call into the one
it() block that uses them. Both shapes are value-preserving to relocate:
codeBlock strips the common leading indentation, so the uniformly
re-indented templates produce identical strings, and Fixtures.get()
resolves relative to the spec file, not the call site's position in it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(lint): enforce inline-single-use-fixtures as error
Narrow the rule to the initializer shapes where moving the declaration
into the test body is provably value-preserving and unambiguously an
improvement: codeBlock/stripIndent tagged templates (indentation-
normalizing, so re-indenting is safe) and Fixtures accessor calls
(resolve relative to the spec file). Plain object/array literals,
untagged template literals (whose leading-newline formatting would need
ugly re-indentation) and arbitrary call expressions remain per-site
judgment calls and are no longer flagged.
With the rule reduced to true defects and all remaining violations
inlined, bump it from warn to error in the spec override block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: inline contents of fixtures relocated into single-use tests
The previous commits moved single-use module-scope Fixtures.get() consts
into their one it() block, which only relocated the file indirection.
Replace those relocated Fixtures.get() calls with the fixture file's
actual content — codeBlock tagged templates for text fixtures — so the
tests are self-contained, and delete the fixture files that no longer
have any references. Relocated fixtures too large to inline comfortably
(terraform-provider releases, sbt private-variable file, nuget v2 pkg
list, bundled package-lock) keep their Fixtures.get() calls.
Value preservation was verified by unchanged vitest results for every
touched spec; all consumers parse the content, so codeBlock's trailing
newline normalization is behavior-neutral.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: inline remaining large single-use fixtures
Inline the four Fixtures.get() sites that were previously exempted for
size, so every single-use fixture is now inlined regardless of size:
- nuget v2_withoutProjectUrl.xml -> codeBlock (served via http mock)
- terraform-provider releaseBackendIndexGoogleBeta.json -> plain object
literal passed to the nock reply
- npm locked-dependency bundled.package-lock.json -> codeBlock JSON
string (consumed as raw lockFileContent)
- sbt private-variable-dependency-file.scala -> codeBlock
All consumers parse the content, so codeBlock's trailing-newline
normalization is behavior-neutral; vitest results are unchanged for
every touched spec, including the two pre-existing nuget failures.
The nuget, terraform-provider and sbt fixture files have no remaining
references and are deleted. bundled.package-lock.json is kept because
package-lock/get-locked.spec.ts still reads it via Fixtures.getJson().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: keep large bundled package-lock fixture in a file
The bundled package-lock.json inlined into the "fails remediation if
bundled" test is ~150 KB (~4,400 lines); inlining it as a codeBlock
hurts readability more than it helps. Read it via Fixtures.get inside
the test block instead — the inline-single-use-fixtures rule permits a
Fixtures.get() call in test scope, and the fixture file is still present.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* refactor(datasource/orb): use REST API instead of graphql-unstable
Switch the orb datasource from the `graphql-unstable` GraphQL endpoint to
the stable CircleCI REST API (`GET /api/v3/orb/packages?filter[name]=`).
The single REST call returns the same fields the datasource needs
(`is_private`, `home_url`, and the nested `orb_versions` with
`created_at`), so behaviour is unchanged for public and private orbs on
circleci.com. Missing orbs are now detected via an empty `data` array
rather than a null `orb` field.
* convert to zod schema
* refactor(datasource/orb): use nullish helper and schema-side transform
Address review feedback:
- use the Nullish schema helper for optional attributes
- catch([]) so orb_versions is never null/undefined
- move the data transform (isPrivate, releases) into the schema; the
homepage fallback stays in the datasource since it needs the requested
packageName
* refactor(datasource/orb): drop snapshot and use toEqual assertion
* refactor(datasource/orb): change assertion to deepnullish
Co-authored-by: Sebastian Poxhofer <secustor@users.noreply.github.com>
---------
Co-authored-by: Sebastian Poxhofer <secustor@users.noreply.github.com>
We've previously been incorrectly caching the error responses from the
Go proxy occurs, which leads to a lot of "flapping" of PRs on
Mend-hosted apps, and can be rather frustrating as a user (and a service
owner).
Per the Go proxy protocol[0], a `,` fallback strategy should only
continue to the next entry on 404/410, and otherwise propagate the
error.
However, we currently break out of the loop and return `result` (which
may be `null` if no successful requests have been performed yet).
This `null` would be cached, leading to a "no releases found"
(`no-result`), stored in the "hard TTL" cache, which could lead to many
days without an update for given packages.
To fix this, we can make sure that we propagate errors, which would
mean that on an error:
- if we've previously successfully retrieved the package, serve the
"last good" result
- if we haven't, it's a full error, and the datasource can retry
[0]: https://go.dev/ref/mod#goproxy-protocol
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
As part of follow-up changes, we'll fix a bug with how the caching of
error responses from the Go proxy occurs, which leads to a lot of
"flapping" of PRs on Mend-hosted apps, and can be rather frustrating as
a user (and a service owner).
Per the Go proxy protocol[0], a `,` fallback strategy should only
continue to the next entry on 404/410, and otherwise propagate the
error.
However, we currently break out of the loop and return `result` (which
may be `null` if no successful requests have been performed yet).
Before we fix our behaviour, we need to clarify what our current
behaviour is.
[0]: https://go.dev/ref/mod#goproxy-protocol
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
The conda datasource has returned `releaseTimestamp` since it was added,
for both the Anaconda.org API (`upload_time`) and prefix.dev
(`createdAt`), but never declared `releaseTimestampSupport`. The docs
therefore listed it as unsupported in both the datasource table and the
`minimumReleaseAge` support table.
Refs #45216
The object counterpart to `prefer-coerce-array`: `x ?? {}` becomes
`coerceObject(x)`. The rule flags a `??` whose default is a bare `{}`
literal, so a populated default or an asserted `({} as Foo)` still
passes; a `x ?? y ?? {}` chain maps onto `coerceObject(x, y)`.
Every hit across `lib/` converted cleanly, so no disable comments were
needed.
* refactor(linters): add prefer-coerce-array rule
Refs #45217
`x ?? []` and `coerceArray(x)` say the same thing, but only the second
one is searchable and consistent. The rule flags a `??` whose default is
a bare `[]` literal; a non-empty default or an asserted `([] as Foo[])`
still passes.
Applying it across `lib/` surfaced three spots the helper cannot cover,
which keep `?? []` with a disable comment or a narrower fix:
- `package-files.ts` destructures a tuple, which `coerceArray()` widens
- `generate.spec.ts` defaults `matchAll()`, an iterator rather than an array
- `mise/upgradeable-tooling.ts` declares a `Record` return, so its
`?? []` fallback was wrong to begin with and becomes `coerceObject()`
* refactor(platform/github): hoist the GHE version header lookup
`Object.entries(...).find()` returns a tuple, and defaulting it with
`?? []` only to destructure the second element reads as an array
default when it is not one. Name the found entry instead, which drops
the `?? []` rather than routing a tuple through `coerceArray()`.
* test: replace manual process.env mutation with vi.stubEnv
Enable Vitest's `unstubEnvs` so env stubs are reverted automatically
before every test, and convert the manual `process.env.X = ...` /
`delete process.env.X` pairs in specs to `vi.stubEnv()`.
This removes the hand-written `afterEach` cleanup blocks and the
`const OLD_ENV = process.env` / `process.env = { ...OLD_ENV }`
save-restore dance, which silently leaked state between tests when a
test set a variable without a matching delete.
Left as-is (incompatible with `unstubEnvs`, which unstubs before each
test, not after):
- module top-level env writes (cocoapods, gomod, post-update, pip-compile
and bundler artifacts specs)
- `beforeAll` env writes -- the two in `util/http` moved to `beforeEach`
* test: stop asserting on ambient environment variables
Three specs asserted on values that the surrounding environment can
override, so they only passed on a machine that happened not to set them:
- `util/exec/env`: `getChildProcessEnv()` forwards every entry of
`basicEnvVars`, but the spec only seeded a subset. Any other forwarded
variable that was set ambiently (e.g. a lowercase `http_proxy`) leaked
into the `toEqual` assertion. Clear all of them first.
- `util/git/index`: `process.env.GIT_SSH_COMMAND` deliberately overrides
the default that the two env-inheritance tests assert on. Clear it.
- `logger/pretty-stdout`: the colorized level strings are built at import
time, so the `beforeEach` setting `FORCE_COLOR` could never affect them
-- it was dead code, and the test silently assumed a terminal without
colour support. Derive the expectation the same way the module does.
* test(datasource/nuget): use a throwaway cache dir for nupkg downloads
The `determine source URL from nupkg` tests do not mock the filesystem,
so they really wrote downloaded packages into a fixed `/tmp/cache` path.
That fails wherever the system tmpdir is not writable, and two checkouts
running the suite at once would share the same directory.
Use `tmp.dir({ unsafeCleanup: true })` and clean it up afterwards, which
is the pattern the other specs that touch disk already use.
* feat(lint): add renovate/prefer-stub-env rule
Enforces in specs what the migration to `vi.stubEnv` established:
- assigning to or deleting `process.env.X` inside `beforeEach`, `afterEach`,
`it` or `test` is reported, with a fixer that rewrites it to `vi.stubEnv()`
- module scope, `describe` bodies and `beforeAll` / `afterAll` are left alone:
`unstubEnvs` reverts stubs *before* each test, so a stub created there is
gone by the time the first test runs, and those must keep assigning directly
- replacing the `process.env` object is reported everywhere and never fixed.
`vi.stubEnv()` captures the original object when the worker starts, so after
a reassignment its deletes silently target the old object
The rule found 28 further violations in specs the earlier sweep missed, since
that one was anchored on `delete process.env`. Twelve were auto-fixed; the
rest replaced `process.env` wholesale and are now `clearEnv()` from test/util.
`util/exec/index.spec.ts` opts out with a comment: it asserts on the whole
child env, and `vi.stubEnv()` cannot express that because it maps `PROD`,
`DEV` and `SSR` onto '' instead of deleting them.
feat(lint): add `types-location` rule and enable it for `lib/modules`
Reviewers consistently ask for exported types to be moved into a
colocated `types.ts`. Add a `renovate/types-location` oxlint rule that
reports `export interface` / `export type` declarations outside such a
file, and turn it on across `lib/modules`.
Re-exports (`export type { X }`, `export type *`), file-local types, and
values (enums, classes, functions) are left alone. `schema.ts` files and
`schema/` directories are excluded via config, matching how
`zod-schema-location` is configured — inferred types belong next to the
schema they are derived from.
Relocating the 87 existing violations means:
- new `types.ts` files for the modules that had none
- `terragrunt/common.ts` folded into its `types.ts`, as it held nothing
else
- `RangeBound` in `versioning/gradle/compare.ts` un-exported, since only
a file-local interface referenced it
- `bazel-module/parser/fragments.ts` split from its inferred types, which
now live in `parser/types.ts` and infer through a namespace import so
each schema keeps its merged name
* fix(lint): use `parseInt` over `Number`
As a code review comment that comes up every so often, we can add a
linting rule to enforce this check.
This also fixes any existing usages.
As this could be breaking, we can release it as a `fix`.
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! fix(lint): use `parseInt` over `Number`
* fixup! fix(lint): use `parseInt` over `Number`
---------
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
Adds a custom oxlint rule renovate/v8-ignore-no-count that flags the
count form '/* v8 ignore next <N> */'. The count is not honored by
vitest's ast-v8-to-istanbul hint parser, which captures only the
keyword (if/else/next/file); 'next' always exempts exactly the next
AST node. Trailing counts therefore suggest a line range that is never
applied and mislead readers.
The rule is autofixable (drops the count, preserving actual behavior)
and has been applied to all existing occurrences.
As reported in #45357, changes in
116c3d610c caused a regression for cases
where an npm package had `homepage: [...]`.
We should restore the `catch(undefined)` and add a test to make sure
this isn't broken in the future.
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* feat(pypi): add support for JSON-based Simple API (PEP 691)
* Fix some stuff
* Clean some things up
* Fix coverage
* Simplify JSON parse
* Remove useless tests
* Align terminology
* Incorporate some changes from PR #44182
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
* Use isNonEmptyString to filter requires_python
* Address review feedback
* Warn once actually
* fixup! review comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! fixup! review comments
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! review comments
As the list of status codes we retry on was based on an unverified
claim that Artifactory, Nexus and devpi answer a rejected `Accept`
header with `400`/`403`/`415`, we should only retry on `406`, which is
what RFC 9110 specifies for a request whose `Accept` cannot be
satisfied.
Our `Accept` header ends with `text/html; q=0.01`, so any conforming
registry can satisfy it, `415` refers to the request body's
`Content-Type` rather than `Accept`, and retrying on `403` doubles the
requests for a registry we're not authenticated against.
The unwrapping of `ExternalHostError` is unaffected, as that's needed
for the `abortOnError` case regardless of which status codes we retry.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! review comments
As the note about `upload-time` sat in the paragraph about the Simple
API, it read as if `minimumReleaseAge` needs a registry which supports
the JSON-based Simple API (PEP 691).
That's only true of the Simple API. The JSON API has always exposed
`upload_time`, and as the default registry of `https://pypi.org/pypi/`
doesn't end in `/simple/`, that's the lookup most users get, so
`minimumReleaseAge` already works for them.
We can split the two apart, so that the guarantee for each API is
stated separately.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* reword
* fixup! review comments
As the JSON-based Simple API's release timestamps were only covered by
hand-written responses, we didn't have the equivalent of the real world
coverage added for the JSON API in
e942788539.
We can add a `numpy` fixture from PyPI's JSON-based Simple API, trimmed
to 1.5.0 and 1.5.1, so we assert against a response a registry has
actually served. 1.5.1 is the interesting case, as it was released in
2010 but gained a wheel in 2014 which PyPI lists first.
This also covers the `upload-time` format the Simple API uses, of
`2010-11-18T14:16:58.801332Z`, rather than the JSON API's
`2010-11-18T14:16:58`, so that we know `asTimestamp` normalises both.
Unlike the JSON API, there's no previous behaviour to characterise
first, as before these changes the Simple API exposed no timestamps at
all.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! fixup! review comments
* fixups
* fixups
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
---------
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <claude-code@jamietanna.co.uk>
* refactor(java-version): extract adoptium HTTP logic from datasource class
Move getPageReleases and the Adoptium fetch logic out of the
JavaVersionDatasource class into a standalone adoptium.ts module.
This makes the code easier to test in isolation and decouples
Adoptium-specific logic from the datasource class.
No behavior change — customRegistrySupport remains false, and the
hardcoded adoptiumRegistryUrl replaces the previously injected
defaultRegistryUrl (which was always the same value).
* fix(java-version): remove unused adoptiumRegistryUrl import from test
* test(java-version): use partial helper in spec
* refactor(java-version): extract adoptium HTTP logic from datasource class
* fix(java-version): pass Adoptium schema to HTTP client
* test(java-version): move Adoptium test to module spec
* chore(ci): retrigger failed checks
As noted in #45314, there are cases where we may see as similar case to
the GitHub Releases datasource in #43451, where a Go module's published
`Time` matches the time that the commit the Git tag points to was
committed, rather than the time that the release was created.
If we do have a GitHub repository, we can look up the GitHub Releases
(or we'll get the Git Tags, if no Releases are present) and use the date
if it's /newer/
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* test(datasource/pypi): cover which `upload_time` a version reports
As part of future changes to how we calculate the `releaseTimestamp`, it
was clear we don't have as much coverage of this logic.
To make this more explicit, we can add new test coverage.
Additionally, we can introduce a new example from a package to guide
this.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fix(datasource/pypi): use the earliest `upload_time` of a version
When calculating the `releaseTimestamp` for the PyPI datasource, we've
been looking at the first entry in the response, which isn't always
the first time this package existed.
In the case that a Wheel is uploaded after the package first was
released - for instance, to support a new version of Python, at an
existing pinned version of the package - this would lead to a
potentially incorrect calculation for the `releaseTimestamp`.
This can lead to Renovate seeing a package as newer than it is, which
can mean that i.e. `minimumReleaseAge` can show as pending for a package
that shouldn't be.
As part of upstream changes[0], this should be less frequent in the
ecosystem, but existing packages still exist with this behaviour.
To fix this, we can make sure that we filter a given Release's
to find the earliest `upload_time`.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
[0]: https://blog.pypi.org/posts/2026-07-22-releases-now-reject-new-files-after-14-days/
---------
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
test(datasource/pypi): add regression
As per changes suggested by Claude Opus 5 in a commit in #44222, a
regression could be triggered with our handling of
`constraintsFiltering=strict`, breaking a key piece of functionality.
To prevent this, we can add a regression test for this functionality.
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fix(datasource/npm): handle absent `homepage`s
As a follow-up to 89f526e0b1, this handles
it at the response from the npm registry.
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! fix(datasource/npm): handle absent `homepage`s
* fixup! fix(datasource/npm): handle absent `homepage`s
---------
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* ci(linter): add renovate/require-regex-util rule
Adds a custom oxlint rule that flags raw RegExp usage in lib/
production code: any `new RegExp(...)` call and any regex literal
that is not the direct first argument of `regEx(...)`. The regEx()
helper from lib/util/regex.ts uses the RE2 engine for ReDoS-safe
matching and caches compiled patterns. lib/util/regex.ts itself is
exempt.
The rule is disabled for **/*.spec.ts, test/**, and tools/** via
config overrides, because vitest APIs such as expect.stringMatching
need bare RegExp instances (a past commit reverted regEx() there).
Severity is set to "warn" instead of "error": the repo currently has
197 pre-existing violations (above the agreed mechanical-fix
threshold of 80), and a number of them are deliberately
RE2-incompatible patterns (lookahead/lookbehind/named backreferences,
see TODO #12872/#12875 markers in e.g. lib/modules/versioning/rez/
pattern.ts and lib/modules/manager/dockerfile/extract.ts) that need
per-site judgment before they can be wrapped in regEx() or suppressed
with a justified disable comment. No violation fixes are included in
this commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci(linter): enforce renovate/require-regex-util at error
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: use regEx() for regex literals and RegExp construction in lib
Burn down all renovate/require-regex-util violations (197):
- wrap RE2-compatible regex literals and RegExp constructions in regEx()
- rewrite lookaround patterns where behavior is provably preserved:
goproxy separator split, sbt-package link extraction, leiningen
comment stripping and keyword search, npm GitHub owner validation
(unit-count bound replaced by an explicit length check)
- keep native RegExp with disable comments where RE2 cannot be used:
moo / good-enough-parser lexer patterns (moo rejects RE2 instances),
MigrationsService propertyName (matched via isRegExp, which RE2
instances fail), rez named backreferences and lookaheads, GitHub
markdown link negative lookbehind, emoji astral-plane matching
- drop stale TODO #12875 comments for patterns current re2 handles
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: reduce require-regex-util disable comments
Rewrite the github massage-markdown-links URL regex to capture an
optional `api.` prefix in a named group (skipped in code) instead of
relying on a negative lookbehind, so both the base and global variants
can use regEx()/RE2 and their two disable comments are removed. Add
spec cases for api-prefixed URLs that match the URL shape.
Consolidate the four related line disables in rez/pattern.ts (bound and
range patterns emulating Python conditional patterns via named
backreferences and lookaheads, which RE2 cannot express) into a single
block disable/enable pair, matching the treatment of the lexer files.
Disable comment count drops from 13 to 8.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(datasource/sbt-package): use named capture group in link regex
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(manager/cake): restore original multiLineComment regex spelling
The [^] -> [\\s\\S] rewrite was only needed for an RE2 wrap that was
later reverted; keep the diff minimal.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(datasource/go): skip empty GOPROXY segments
The switch from a lookahead split to a capture-group split (needed because
RE2 has no lookahead support) changed how empty GOPROXY segments parse:
`a|,b` yielded `{ url: ',', fallback: '|' }` before and
`{ url: '', fallback: ',' }` after. The comma fallback only continues on
404/410, so a failing proxy ahead of an empty segment silently stopped the
lookup instead of falling through to the remaining proxies.
Empty segments carry no url to query, so drop them.
* refactor(datasource/go): use isTruthy for empty segment filter
* fix(datasource/go): filter empty segments before mapping
Filtering after the map broke the contextual typing from the
GoproxyItem[] annotation, widening fallback to string.
* refactor(platform/github): use regEx() for merge queue query stripping
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Jamie Tanna