To harden Renovate from being able to access internal hosts, we can add
a level of enforcement on all outgoing requests.
We can split this into two types of traffic:
- where the response becomes configuration (i.e. `extends`)
- where the response does not become configuration (i.e.
`customDatasources`)
To harden this, we enforce on every outbound HTTP request:
- check the resolved URL before anything, including cache lookups, can
proceed
- catching IP-literal hosts in every spelling the WHATWG URL parser normalizes
- blocking known Cloud metadata IPs, too
- a `dnsLookup` wrapper validates every address a hostname resolves to,
on every new connection, which also defeats DNS rebinding
- a `beforeRedirect` hook re-runs the URL check on each redirect
target, with the new host getting its own grant lookup rather than
inheriting the original host's verdict
We default this enforcement to `warn`, to not break existing deployments
and instead give them visibility into required configuration before a
future major version (major+1) enables `block` by default.
We'll also remove allow in a future major version (major+2), to give
time for admins to migrate over.
When a request is blocked, this degrades like an unreachable registry,
so a Renovate run isn't aborted by a blocked request.
As part of this, we can also provide a worked example of the different
cases that do/do not need allowing via `hostRules`.
Requests through an admin-configured proxy resolve DNS at the proxy, so
`dnsLookup` never runs for them, so we can resolve the hostname locally
and validate every address before the request instead.
This doesn't catch everything, especially as the proxy may resolve the
name to a different result, and if we can't resolve the name - possible
in locked-down corporate environments - we fail open.
Child processes aren't currently able to route traffic through
Renovate's HTTP layer, so are unrestricted.
Co-authored-by: Claude Fable 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fix(platform/github): skip PR update when PR is in merge queue
Updating a PR that is queued for merging can remove it from the merge
queue. Check the PR's merge queue status via GraphQL before updating
and skip the update if it is queued. The check is skipped on GHES
<3.12.0 and fails open on errors.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(platform/github): dequeue PR from merge queue instead of skipping update
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(platform/github): note re-queue behavior after dequeue
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(platform/github): dequeue merge queue PR in commitAndPush
Dequeue on push instead of on PR metadata update, since pushing to the
branch is what invalidates the queued merge result.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(platform/github): extract dequeuePr to pr.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(platform/github): abort branch update when PR is in merge queue
Instead of dequeueing the PR, throw PR_ALREADY_IN_MERGE_QUEUE from
commitAndPush and handle it gracefully in processBranch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(platform/github): check merge queue enablement per base branch
Cache merge queue enablement per base branch like branchForceRebase and
skip the per-PR merge queue check when the base branch has no merge
queue. Errors are treated as enabled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(platform/github): fetch default branch merge queue status in initRepo
Include mergeQueue in repoInfoQuery so no extra query is needed for the
default branch, stripping it on GHES <3.12.0 like other unsupported
fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* - Added Renovate support for the [SCM-Manager](https://scm-manager.org/)
- SCM-Manager is a repository management tool like GitHub
- It is maintained as an open source project by the company [Cloudogu](https://cloudogu.com/en/)
- The SCM-Manager support is not feature complete yet, features like auto merging are still missing
- The SCM-Manager also got added to the documentation as another platform
author Thomas Zerr <thomas.zerr@cloudogu.com> 1690877165 +0200
committer Eduard Heimbuch <eduard.heimbuch@cloudogu.com> 1706264263 +0100
* Reverted numbering change in markdown docs
* Changed from type to interface, if possible
* Removed axios dependency and using the http layer of renovate instead
* Fixed not using scmm specific mime types for accept header
* Use git from test/util instead
* Add dummy tests for no ops
* Fix some phrasing and spelling issues
* Change scmm to scm-manager and fix SCM-Manager specific content type
* Add supported major versions, review plugin requirement, write permissions is needed
* Change using actual host rules instead of mocking it
* Change use http mocks for testing
* Update docs/usage/getting-started/running.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/index.ts
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/mapper.spec.ts
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix scm-manager docs link
* Fix linting
* Fix linting
* Throw errors if username or token is not provided by the host rules
* Replace NO-OP with Not implemented
* Remove objects from log messages
* Fix linting
* Fix path to platform config options
* Fix link to self-host-configuration docs
* Fix link to self-host-configuration docs
* Remove the link to the platform config option, because the tests keep failing
* Fix link to self-host-configuration docs
* Remove not needed type cast
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Fix type check of protocol links
* Move scm-manager into util http folder
* Apply linting
* Add maxBodyLength function and its test
* Add test case if protocol links are not an array of links
* Use zod schema for pagination
* Apply linting
* Refactor typing with zod schema
* Fix test description
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix test description
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix test description
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix test description
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update readme
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Add try catch around fetching the current user
* Switch log statement from info to debug
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from info to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from debug to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from info to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from info to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from info to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Fix linting errors
* Remove credentials check in initRepo
* Fix imports
* Fix zod schema for repositories
* Fix zod schema for pull requests
* Fix 500 error after attempting to update an pr
* Fix wrong typing
* Refactor parameterized tests for better readability
* Refactor SCM HTTP API
* Add setToken functionality
* Remove not needed type definition
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
* Fix assertion
* Fix istanbul comment
* Change log statement
* Remove type rename
* Move inferred types close to corresponding schemas
* Add platform is experimental
* Remove type renaming
* Fix type imports
* Fix test suite description
* Remove debug log statement
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
* Fix warning
* Fix paremeterized unit tests with string templates
* Fix spelling
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix eslint error
* Convert trace logs to debug logs
* Convert info logs to debug
* Replace manual null and undefinded check with is util
* Rephrase instruction in readme
* Use ensureTrailingSlash instead
* Fix linting
* Keep type and schema together
* Simplify hasAssignees check
* Simplify reviewer mapping
* Remove istanbul comments
* Replace custom scmm header option with renovate native http headers option
* Let the token be passed in initPlatform
* Add description which permissions are needed
* Add support for ignorePrAuthor filter
* Add notes about automerge feature within platform readme
* Add support for fast_forward_only
* Change supported versions due to merge strategy change
* Remove commented code
* Fix import
* Use email schema
* Heed review advice
* Fix import and empty array assertions
* Fix linting issue
* Fix import errors
* Fix removed ignorePrAuthor parameter from initRepo function
* Fix failing unit tests by setting/reseting the global config
* Mark scm-manager as experimental
* Fix zod import path
* Change endpoint configuration, so that user only needs to define origin, but not also the path
* Fix warning of redundant backslash
---------
Co-authored-by: Eduard Heimbuch <eduard.heimbuch@cloudogu.com>
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
Co-authored-by: René Pfeuffer <rene@pfeuffer-web.de>
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: René Pfeuffer <rene.pfeuffer@cloudogu.com>
As a way to make it more predictable what the possible states the
`result` in a given `Repository finished` log message can be, we can
wrap these into a type.
This requires a slight refactor where these are used to make sure that
Typescript is happy with the value being returned.
We can capture the common groups of errors (which are then present as a
`RepositoryResult`) and expose that to be used.
feat(hostRules): automagically use host rules for any `api.github.com` lookups
As noted in #38732, the Typst datasource performs API calls to the
GitHub APIs, but as we do not have explicit datasource mapping for the
API calls (via `GITHUB_API_USING_HOST_TYPES`), we end up sending
unauthenticated API calls, which then can result in rate limits.
We had previously attempted to fix this in #30490, for a different
datasource, and there are likely other datasources impacted.
To simplify this, we can auto-wire any `https://api.github.com/` API
calls with the `hostType: github` hostRule, when matching against the
hostname from the `url`.
This also adds a note to deprecate the usage of
`GITHUB_API_USING_HOST_TYPES`, unless it's explicitly needed.
Closes#38953.
* feat: add datasource for Hermit package manager
* fix: make hermit datasource use http.stream instead of http.get
* chore: inline small function and return null on error in hermit datasource
* fix: use regExp to extract owner and repo from registryUrl
* fix: use http from super & use error instead of debug on datasource error
* fix: fix type check failure for github-release test
* Update lib/modules/datasource/hermit/index.spec.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Update lib/modules/datasource/hermit/index.spec.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Update lib/modules/datasource/hermit/index.spec.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Update lib/modules/datasource/hermit/index.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* fix: wrap JSON.parse to avoid breaking process, plus more comment onto the search manifest streaming process
* fix: add test to cover the invalid json case
* fix: change some logger.errors to logger.warn in hermit datasource, error will exit the whole process
* Update lib/modules/datasource/hermit/index.spec.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Update lib/modules/datasource/hermit/index.spec.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Update lib/modules/datasource/hermit/index.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Update lib/modules/datasource/hermit/index.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Update lib/modules/datasource/hermit/index.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* fix: use parseUrl and use hermit as GithubHttp id
* fix: added test to include HermitDatasource type in GITHUB_API_USING_HOST_TYPES
* fix: handle invalid url & achieve code coverage
* fix: move url parsing to parent function
* Update lib/modules/datasource/hermit/index.ts
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* chore: clean up HermitDataSource class doc
* fix: getHermitSearchManifest cache key function should expect undefined
* fix: pass GetReleasesConfig into getHermitSearchManifest to get proper caching working
* Update lib/modules/datasource/hermit/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/datasource/hermit/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/datasource/hermit/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/datasource/hermit/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/datasource/hermit/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* chore: fix json block in datasource readme.md
* Put packageRules example in step 4
* fix: fix lint error
* fix: pass parsedUrl into getHermitSearchManifest, add warning and simplifies type
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
MFietze