`lib/util/http/auth.ts` and `lib/util/http/host-rules.ts` were the last
readers of the `*_API_USING_HOST_TYPES` constants, which the previous
commits left as aliases of `PLATFORM_FAMILIES`. Both now read the table
directly and the aliases are deleted, so each family's host types are
declared in exactly one place.
Also covers the `forgejo` fallback in `findMatchingRule()`, which had no
test: the shared setup only registers a `gitea` host rule, so the
forgejo branch was never entered.
Behaviour changes: none. The aliases pointed at the same arrays.
* refactor(constants): add a platform family capability table
Every consumer of `detectPlatform` carried its own mapping from the
detected platform to a tags datasource or a web UI path, and the seven
`*_API_USING_HOST_TYPES` arrays were checked one by one to produce that
platform in the first place.
Introduce `PLATFORM_FAMILIES`, one record keyed by family holding the
host types, the hostnames which identify the family outright, the tags
datasource, the API base URL derivation and the web UI path to a
directory at the default branch. A family is a group of platforms, so
the ids are checked against `PlatformId`: a family which is not a
platform is a compile error. Not every platform is a family, because
codecommit, gerrit, local and scm-manager are never recognized from a
URL.
`detectPlatform` loops the table for both its hostname and its hostType
matching, and the consumers look their answers up:
- `github-actions` and `pre-commit` read `tagsDatasource` instead of
importing the tags datasource classes.
- The PR body reads `webDirPath` instead of special-casing Bitbucket and
Bitbucket Server.
`apiBaseUrl` has no consumer yet; the changelog sources pick it up in
the next commit.
`lib/util/http/auth.ts` and `lib/util/http/host-rules.ts` still read six
of the old arrays by name, so those stay as aliases of the table and are
migrated in a follow-up. `AZURE_API_USING_HOST_TYPES` had no other
consumer and is dropped.
Behaviour changes: none. `detectPlatform` now tests every known hostname
before its name heuristics instead of interleaving the two, which is
equivalent because the hostname lists are pairwise disjoint and no known
hostname contains the name of another family.
* Apply suggestion from @secustor
To harden Renovate from being able to access internal hosts, we can add
a level of enforcement on all outgoing requests.
We can split this into two types of traffic:
- where the response becomes configuration (i.e. `extends`)
- where the response does not become configuration (i.e.
`customDatasources`)
To harden this, we enforce on every outbound HTTP request:
- check the resolved URL before anything, including cache lookups, can
proceed
- catching IP-literal hosts in every spelling the WHATWG URL parser normalizes
- blocking known Cloud metadata IPs, too
- a `dnsLookup` wrapper validates every address a hostname resolves to,
on every new connection, which also defeats DNS rebinding
- a `beforeRedirect` hook re-runs the URL check on each redirect
target, with the new host getting its own grant lookup rather than
inheriting the original host's verdict
We default this enforcement to `warn`, to not break existing deployments
and instead give them visibility into required configuration before a
future major version (major+1) enables `block` by default.
We'll also remove allow in a future major version (major+2), to give
time for admins to migrate over.
When a request is blocked, this degrades like an unreachable registry,
so a Renovate run isn't aborted by a blocked request.
As part of this, we can also provide a worked example of the different
cases that do/do not need allowing via `hostRules`.
Requests through an admin-configured proxy resolve DNS at the proxy, so
`dnsLookup` never runs for them, so we can resolve the hostname locally
and validate every address before the request instead.
This doesn't catch everything, especially as the proxy may resolve the
name to a different result, and if we can't resolve the name - possible
in locked-down corporate environments - we fail open.
Child processes aren't currently able to route traffic through
Renovate's HTTP layer, so are unrestricted.
Co-authored-by: Claude Fable 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fix(platform/github): skip PR update when PR is in merge queue
Updating a PR that is queued for merging can remove it from the merge
queue. Check the PR's merge queue status via GraphQL before updating
and skip the update if it is queued. The check is skipped on GHES
<3.12.0 and fails open on errors.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(platform/github): dequeue PR from merge queue instead of skipping update
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(platform/github): note re-queue behavior after dequeue
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(platform/github): dequeue merge queue PR in commitAndPush
Dequeue on push instead of on PR metadata update, since pushing to the
branch is what invalidates the queued merge result.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(platform/github): extract dequeuePr to pr.ts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(platform/github): abort branch update when PR is in merge queue
Instead of dequeueing the PR, throw PR_ALREADY_IN_MERGE_QUEUE from
commitAndPush and handle it gracefully in processBranch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(platform/github): check merge queue enablement per base branch
Cache merge queue enablement per base branch like branchForceRebase and
skip the per-PR merge queue check when the base branch has no merge
queue. Errors are treated as enabled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(platform/github): fetch default branch merge queue status in initRepo
Include mergeQueue in repoInfoQuery so no extra query is needed for the
default branch, stripping it on GHES <3.12.0 like other unsupported
fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* - Added Renovate support for the [SCM-Manager](https://scm-manager.org/)
- SCM-Manager is a repository management tool like GitHub
- It is maintained as an open source project by the company [Cloudogu](https://cloudogu.com/en/)
- The SCM-Manager support is not feature complete yet, features like auto merging are still missing
- The SCM-Manager also got added to the documentation as another platform
author Thomas Zerr <thomas.zerr@cloudogu.com> 1690877165 +0200
committer Eduard Heimbuch <eduard.heimbuch@cloudogu.com> 1706264263 +0100
* Reverted numbering change in markdown docs
* Changed from type to interface, if possible
* Removed axios dependency and using the http layer of renovate instead
* Fixed not using scmm specific mime types for accept header
* Use git from test/util instead
* Add dummy tests for no ops
* Fix some phrasing and spelling issues
* Change scmm to scm-manager and fix SCM-Manager specific content type
* Add supported major versions, review plugin requirement, write permissions is needed
* Change using actual host rules instead of mocking it
* Change use http mocks for testing
* Update docs/usage/getting-started/running.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/index.ts
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/mapper.spec.ts
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update lib/modules/platform/scm-manager/readme.md
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix scm-manager docs link
* Fix linting
* Fix linting
* Throw errors if username or token is not provided by the host rules
* Replace NO-OP with Not implemented
* Remove objects from log messages
* Fix linting
* Fix path to platform config options
* Fix link to self-host-configuration docs
* Fix link to self-host-configuration docs
* Remove the link to the platform config option, because the tests keep failing
* Fix link to self-host-configuration docs
* Remove not needed type cast
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Fix type check of protocol links
* Move scm-manager into util http folder
* Apply linting
* Add maxBodyLength function and its test
* Add test case if protocol links are not an array of links
* Use zod schema for pagination
* Apply linting
* Refactor typing with zod schema
* Fix test description
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix test description
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix test description
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix test description
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Update readme
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Add try catch around fetching the current user
* Switch log statement from info to debug
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from info to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from debug to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from info to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from info to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Switch log statement from info to trace
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
* Fix linting errors
* Remove credentials check in initRepo
* Fix imports
* Fix zod schema for repositories
* Fix zod schema for pull requests
* Fix 500 error after attempting to update an pr
* Fix wrong typing
* Refactor parameterized tests for better readability
* Refactor SCM HTTP API
* Add setToken functionality
* Remove not needed type definition
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
* Fix assertion
* Fix istanbul comment
* Change log statement
* Remove type rename
* Move inferred types close to corresponding schemas
* Add platform is experimental
* Remove type renaming
* Fix type imports
* Fix test suite description
* Remove debug log statement
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
* Fix warning
* Fix paremeterized unit tests with string templates
* Fix spelling
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
* Fix eslint error
* Convert trace logs to debug logs
* Convert info logs to debug
* Replace manual null and undefinded check with is util
* Rephrase instruction in readme
* Use ensureTrailingSlash instead
* Fix linting
* Keep type and schema together
* Simplify hasAssignees check
* Simplify reviewer mapping
* Remove istanbul comments
* Replace custom scmm header option with renovate native http headers option
* Let the token be passed in initPlatform
* Add description which permissions are needed
* Add support for ignorePrAuthor filter
* Add notes about automerge feature within platform readme
* Add support for fast_forward_only
* Change supported versions due to merge strategy change
* Remove commented code
* Fix import
* Use email schema
* Heed review advice
* Fix import and empty array assertions
* Fix linting issue
* Fix import errors
* Fix removed ignorePrAuthor parameter from initRepo function
* Fix failing unit tests by setting/reseting the global config
* Mark scm-manager as experimental
* Fix zod import path
* Change endpoint configuration, so that user only needs to define origin, but not also the path
* Fix warning of redundant backslash
---------
Co-authored-by: Eduard Heimbuch <eduard.heimbuch@cloudogu.com>
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
Co-authored-by: René Pfeuffer <rene@pfeuffer-web.de>
Co-authored-by: Michael Kriese <michael.kriese@gmx.de>
Co-authored-by: René Pfeuffer <rene.pfeuffer@cloudogu.com>
As a way to make it more predictable what the possible states the
`result` in a given `Repository finished` log message can be, we can
wrap these into a type.
This requires a slight refactor where these are used to make sure that
Typescript is happy with the value being returned.
We can capture the common groups of errors (which are then present as a
`RepositoryResult`) and expose that to be used.
feat(hostRules): automagically use host rules for any `api.github.com` lookups
As noted in #38732, the Typst datasource performs API calls to the
GitHub APIs, but as we do not have explicit datasource mapping for the
API calls (via `GITHUB_API_USING_HOST_TYPES`), we end up sending
unauthenticated API calls, which then can result in rate limits.
We had previously attempted to fix this in #30490, for a different
datasource, and there are likely other datasources impacted.
To simplify this, we can auto-wire any `https://api.github.com/` API
calls with the `hostType: github` hostRule, when matching against the
hostname from the `url`.
This also adds a note to deprecate the usage of
`GITHUB_API_USING_HOST_TYPES`, unless it's explicitly needed.
Closes#38953.
Sebastian Poxhofer