To harden Renovate from being able to access internal hosts, we can add
a level of enforcement on all outgoing requests.
We can split this into two types of traffic:
- where the response becomes configuration (i.e. `extends`)
- where the response does not become configuration (i.e.
`customDatasources`)
To harden this, we enforce on every outbound HTTP request:
- check the resolved URL before anything, including cache lookups, can
proceed
- catching IP-literal hosts in every spelling the WHATWG URL parser normalizes
- blocking known Cloud metadata IPs, too
- a `dnsLookup` wrapper validates every address a hostname resolves to,
on every new connection, which also defeats DNS rebinding
- a `beforeRedirect` hook re-runs the URL check on each redirect
target, with the new host getting its own grant lookup rather than
inheriting the original host's verdict
We default this enforcement to `warn`, to not break existing deployments
and instead give them visibility into required configuration before a
future major version (major+1) enables `block` by default.
We'll also remove allow in a future major version (major+2), to give
time for admins to migrate over.
When a request is blocked, this degrades like an unreachable registry,
so a Renovate run isn't aborted by a blocked request.
As part of this, we can also provide a worked example of the different
cases that do/do not need allowing via `hostRules`.
Requests through an admin-configured proxy resolve DNS at the proxy, so
`dnsLookup` never runs for them, so we can resolve the hostname locally
and validate every address before the request instead.
This doesn't catch everything, especially as the proxy may resolve the
name to a different result, and if we can't resolve the name - possible
in locked-down corporate environments - we fail open.
Child processes aren't currently able to route traffic through
Renovate's HTTP layer, so are unrestricted.
Co-authored-by: Claude Fable 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fix(docker): adjust tag caching for private registries
Tag caching for private registries currently has two issues, observed in
issues #45249 and #40734:
1. `getTags` has no `cacheable` guard (unlike `getReleases`)
2. `getTags` caches `null` tag results
These two combined mean that regardless of `cachePrivatePackages`
settings, private registry tag lookups which fail due to transient/auth
errors are cached. This results in the cache being "poisoned" by
misconfigured jobs, with well-configured jobs unable to recover. This is
especially relevant for the hosted app where caching is shared by
different tenants.
This commit:
1. Aligns `getTags` caching with `getReleases`, ensuring caching
behaviour is driven by `cachePrivatePackages`.
2. Prevents `null` tag results being cached, allowing for recovery
regardless of `cachePrivatePackages` configuration.
Note that I have _not_ applied (2) to `getReleases`, because
1. `getReleases` has a slightly different contract in that it sets
`fallback: true` to intentionally cache failures
2. I have not observed any actual failures in `getReleases` due to (20,
so it hasn't been empirically proven as requiring a fix
3. `getTags` is internal so is a lower-risk change
Fixes
- #45249
Relates to
- #45249
- #40734
* refactor(docker): undefined for no-auth
Previously `getAuthHeaders()` used `null` as an auth-failed sentinel,
which was inconsistent with `withCache()`, and led to the original "fix"
where nulls were explicitly omitted from the cache.
Instead this implements the suggestion from https://github.com/renovatebot/renovate/pull/45348#issuecomment-5340524967
to uniformly return undefined for `getTags` auth errors, which allows
the cache to recognise failures itself. Then cacheability ends up being
conditional on dockerhub as always.
This also rectifies a mismatch between auth-failure signals, where other
functions like `_getImageConfig` were returning `undefined` when headers
were unavailable.
IMPORTANT: once this lands, since there aren't any cache invalidations,
the existing issue will still be observed until the cache TTL expires
and old `null` entries are evicted.
* fix: whoops, sort out the vacuous tests
I failed to realise caching isn't automatically orchestrated in these
tests. As an eagle-eyed observer [pointed out][1], this meant the tests
weren't actually exercising anything.
This commit fixes that by adding a new test block which does the cache
orchestration using a temp dir cache. I've also opted to add assertions
against the mock server's http trace counter directly to ensure the mock
really is/isn't being hit. This should also serve to make the tests fail
fast if the underlying impl flow (auth probe -> get tags) is adjusted in
the future.
I've also brought in some caching dependencies into the test file, but
those are used elsewhere too (e.g. `withCache` tests).
This time, I have actually verified the tests which we'd expect to fail
do fail against main. Note that the first 'caches tags for Docker Hub'
is expected to pass on both branches (the control test).
```
❯ getTags caching (3)
✓ caches tags for Docker Hub 21ms 249 MB heap used
× does not cache tags for a private registry 20ms 212 MB heap used
× does not cache failed lookups when cachePrivatePackages is enabled 15ms 215 MB heap used
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Tests 2 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
FAIL lib/modules/datasource/docker/index.spec.ts > modules/datasource/docker/index > getTags caching > does not cache tags for a private registry
AssertionError: expected [ '1.0.0' ] to deeply equal [ '1.0.0', '2.0.0' ]
- Expected
+ Received
[
"1.0.0",
- "2.0.0",
]
❯ lib/modules/datasource/docker/index.spec.ts:3694:72
3692| expect(httpMock.getTrace()).toHaveLength(2);
3693|
3694| expect(await ds.getTags('https://registry.company.com', 'node')).toEqual([
| ^
3695| '1.0.0',
3696| '2.0.0',
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/4]⎯
FAIL lib/modules/datasource/docker/index.spec.ts > modules/datasource/docker/index > getTags caching > does not cache tags for a private registry
Error: *** Unused HTTP mocks ***
- GET https://registry.company.com/v2/node/tags/list
Requests done:
- GET https://registry.company.com/v2/node/tags/list?n=10000 [200]
- GET https://registry.company.com/v2/node/tags/list?n=10000 [200]
---
Renovate testing strategy requires that every HTTP request
has a corresponding mock.
This error occurs because some of the created mocks are unused.
In most cases, you simply need to remove them.
+++
❯ <test> lib/modules/datasource/docker/index.spec.ts:3675:9
3673| });
3674|
3675| it('does not cache tags for a private registry', async () => {
| ^
3676| httpMock
3677| .scope('https://registry.company.com/v2')
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/4]⎯
FAIL lib/modules/datasource/docker/index.spec.ts > modules/datasource/docker/index > getTags caching > does not cache failed lookups when cachePrivatePackages is enabled
AssertionError: expected null to be undefined
- Expected:
undefined
+ Received:
null
❯ lib/modules/datasource/docker/index.spec.ts:3717:9
3715| expect(
3716| await ds.getTags('https://registry.company.com', 'node'),
3717| ).toBeUndefined();
| ^
3718| expect(httpMock.getTrace()).toHaveLength(1);
3719|
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/4]⎯
FAIL lib/modules/datasource/docker/index.spec.ts > modules/datasource/docker/index > getTags caching > does not cache failed lookups when cachePrivatePackages is enabled
Error: *** Unused HTTP mocks ***
- GET https://registry.company.com/v2/node/tags/list
Requests done:
- GET https://registry.company.com/v2/node/tags/list?n=10000 [403]
---
Renovate testing strategy requires that every HTTP request
has a corresponding mock.
This error occurs because some of the created mocks are unused.
In most cases, you simply need to remove them.
+++
❯ <test> lib/modules/datasource/docker/index.spec.ts:3701:9
3699| });
3700|
3701| it('does not cache failed lookups when cachePrivatePackages is enabled', async () => {
| ^
3702| GlobalConfig.set({ cachePrivatePackages: true });
3703|
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/4]⎯
Test Files 1 failed (1)
Tests 2 failed | 313 passed (315)
```
[1]: https://github.com/renovatebot/renovate/pull/45348#pullrequestreview-5092478059
* oxlint-fix
---------
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
Refs #45216
The object counterpart to `prefer-coerce-array`: `x ?? {}` becomes
`coerceObject(x)`. The rule flags a `??` whose default is a bare `{}`
literal, so a populated default or an asserted `({} as Foo)` still
passes; a `x ?? y ?? {}` chain maps onto `coerceObject(x, y)`.
Every hit across `lib/` converted cleanly, so no disable comments were
needed.
Adds a custom oxlint rule renovate/v8-ignore-no-count that flags the
count form '/* v8 ignore next <N> */'. The count is not honored by
vitest's ast-v8-to-istanbul hint parser, which captures only the
keyword (if/else/next/file); 'next' always exempts exactly the next
AST node. Trailing counts therefore suggest a line range that is never
applied and mislead readers.
The rule is autofixable (drops the count, preserving actual behavior)
and has been applied to all existing occurrences.
* ci(linter): add renovate/v8-ignore-reason rule
Flag coverage-ignore comments ("v8 ignore") that carry no justification.
Accepted forms: "-- reason" / "--- reason" (recommended) and the legacy
colon form "v8 ignore <kind>[ <count>]: reason", which is already
widespread in the codebase (29 sites). Bare "v8 ignore stop" markers are
exempt because the justification lives on the paired "start" marker. The
"v8 ignore next <N>" count form remains allowed; this rule only requires
a justification.
Registered as "warn" instead of "error": the repo currently has 119
reason-less v8 ignore comments, too many to fix mechanically in one PR,
so existing sites are left untouched and can be fixed incrementally.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci(linter): promote renovate/v8-ignore-reason to error
Record the empirical verdict on the contested "v8 ignore next <N>" count
semantics in the rule doc: verified against vitest 4 with
ast-v8-to-istanbul that the count is not honored - the ignore hint always
exempts exactly the next AST node regardless of N. Counts are therefore
inert and stay allowed; the rule keeps requiring only a justification.
Promoting to error so all remaining reason-less coverage-ignore comments
must be fixed; the burndown follows in subsequent commits.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor: add justifications to coverage-ignore comments (batch 1)
Give every reason-less "v8 ignore" comment outside the large platform
index files a site-specific justification explaining why the exempted
branch is untestable or defensive, as now required by the
renovate/v8-ignore-reason rule.
Also guard against a nullish comment.loc in the rule itself; the
merged-in stricter tools typechecking flagged the unguarded access.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* refactor: add justifications to coverage-ignore comments (batch 2)
Cover the five large platform index files (github, bitbucket,
bitbucket-server, codecommit, gitlab) with site-specific reasons on
every "v8 ignore" comment, completing the renovate/v8-ignore-reason
burndown. Two meaningless ignore hints inside commented-out CodeCommit
mergePr code are dropped instead of annotated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017RxHFNsSworLcv23xXk9Hk
* Apply suggestions from code review
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
* Update lib/util/regex.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* fix(docker): handle 405 maxResults error on auth probe for ECR-compatible proxies
Some private Docker registries that front ECR reject `n=10000` on the tags
URL even during the auth-probe request, returning a 405 with the ECR-style
"Member must have value less than or equal to 1000" error body.
`getAuthHeaders` had no handler for this case and returned null, so the
lookup aborted before the main fetch loop's existing n=1000 retry could fire.
Fix: extract `isECRMaxResultsResponse` from `isECRMaxResultsError` in ecr.ts
and use it in `getAuthHeaders` to detect the 405 pattern. When matched, fall
back to probing the base `/v2/` endpoint for auth, then let the existing
`isECRMaxResultsError` retry in the tag-fetch loop handle the n=1000 fallback.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(docker): address PR review comments on ECR proxy auth probe fix
- Shorten test token to 'abc' to avoid CodeQL secret-detection false positive
- Replace HttpResponse<any> with HttpResponse<unknown> in common.ts and ecr.ts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(docker): replace HttpResponse<any> with HttpResponse<unknown> in isECRMaxResultsResponse
Cast body to a typed shape inline instead of widening the whole response to any.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(docker): replace toMatchInlineSnapshot with toEqual in ECR auth probe test
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(docker): replace getJsonUnchecked with validated getJson
Replace all getJsonUnchecked calls in the Docker datasource with
Zod-validated getJson calls. Add RegistryAuthToken, RegistryTagsList,
and QuayTagsResponse schemas to schema.ts and add unit tests for them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* do not use getJson if we do not look at the body
* chore: default to false for safety
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(noNewURL): add new rule enforcing parseUrl() from 'lib/util/url.ts'
* replace newURL with parseUrl
* remove non-null assertions marks if appropiate
* test: add additional tests
* test: add additional tests and ignore where it makes sense
* implement suggestions
* test: further improve coverage
* apply suggestion
Moves datasource, manager, platform and versioning code from lib/ into new lib/modules/
BREAKING CHANGE: External tools must update paths to datasource, manager, platform and versioning
Jamie Tanna