1107 Commits

Author SHA1 Message Date
Michael Kriese 71d4f95ecc fix(workers): keep URL credentials of dependencies out of the repository cache and generated texts
Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
Co-Authored-By: Claude Sonnet 5 <michael.kriese+claude-code@mend.io>
2026-10-20 09:05:00 +01:00
Michael Kriese 320f770c36 fix(cache): hash URL credentials in cache keys
Co-Authored-By: Claude Fable 5.1 <michael.kriese+claude-code@mend.io>
Co-Authored-By: Claude Opus 5 <michael.kriese+claude-code@mend.io>
Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
2026-10-20 09:05:00 +01:00
Martin Costello ea1c9a7bf3 feat(datasource/github-runners): add xcode-27 (#47116)
* feat(datasource/github-runners): add xcode-27

Add `xcode-27` and `xcode-27-xlarge` runner labels to the datasource.

* feat(datasource/github-runners): add release date

Add release date for the XCode 27 runners.
2026-10-19 16:59:55 +00:00
Jamie Tanna 15fb543905 feat(datasource/github-runners): add releaseTimestamps (#46885)
* feat(datasource/github-runners): add `releaseTimestamp`s

As per #46806, we can retrospectively add the relevant
`releaseTimestamp`s for the `github-runners` Datasource.

Claude Sonnet has cross-referenced this with announcements [0] with
rough measures from the Git history, and seems confident.

We can also add a test to make sure that all known release channels have
a `releaseTimestamp`, to aid future contributions.

Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>

[0]: https://github.blog/changelog/label/actions/

* fixup! feat(datasource/github-runners): add `releaseTimestamp`s

Co-authored-by: RahulGautamSingh <rahultesnik@gmail.com>

---------

Co-authored-by: RahulGautamSingh <rahultesnik@gmail.com>
2026-10-19 15:51:55 +00:00
Sebastian Poxhofer 0070fc5e6a refactor: remove unused exports and types (#47130)
Found with knip.
2026-10-17 09:10:12 +00:00
Sebastian Poxhofer b8f6dbb83e refactor(datasource): move the git host tags datasources onto GitHostTagsDigestDatasource (#46896)
* refactor(datasource): expose the source URL of the git host tags datasources

getSourceUrl() becomes a public instance method of every git host tags datasource. github-tags and gitlab-tags gain it next to a static getSourceUrl(), and the Gitea base derives it from its default registry.

* refactor(datasource/gitea-tags): extend GitHostTagsDigestDatasource

The Gitea and Forgejo tags and releases datasources keep their cache keys and only cache lookups against the public instance, through a new isCacheable() hook of the base class. The cache key of a releases lookup now ends in ':tags' like every other one, in its own namespace.

* refactor(datasource/gitlab-tags): extend GitHostTagsDigestDatasource

The tags, digests and tag commits are cached under the '<registryUrl>:<repo>:<type>' keys of the base class, so the first lookup after the upgrade misses the package cache.

* refactor(datasource/github-tags): extend GitHostTagsDigestDatasource

getReleases() and getDigest() skip the package cache of the base class, because the GraphQL fetcher already caches the tags with their commits and the HTTP client caches the latest commit in memory. The releases now carry their registryUrl like the other tags datasources.

* fix(datasource): only cache git host tags lookups known to be public

A tags lookup may read a private repository, so GitHostTagsDatasource no longer writes it to the package cache by default. Gitea and Forgejo keep caching their public instance; Bitbucket Cloud, Bitbucket Data Center, Azure DevOps and GitLab lookups are cached only with cachePrivatePackages. The Bitbucket main branch cache follows the same rule.

* test(datasource): cover the default registry of github-tags and gitlab-tags

* docs(datasource/github-tags): drop the cache note from getDigest

* refactor(datasource): keep caching git host tags lookups by default

* refactor(datasource): keep getSourceUrl of the git host tags datasources protected

* refactor(datasource): default getRegistryUrl and resolve the URL for isCacheable

* fix(datasource/gitlab-tags): normalise the registry URL like main

* refactor(datasource): cache tag digests only once

* refactor(datasource/github-tags): skip the package cache through a base hook

* refactor(datasource): document getSourceUrl once in the base class

* refactor(datasource/bitbucket-tags): inline the static registry URL helper

* refactor(datasource): rename usePackageCache to useGitHostTagCaching

* refactor(datasource): drop isCacheableFor and useGitHostTagCaching
2026-10-16 09:52:58 +00:00
Sebastian Poxhofer 7286ebea6b refactor(versioning/gomod): parse the commit of pseudo-versions (#47097) 2026-10-16 09:52:08 +00:00
Jamie Tanna 0876273a37 feat(datasource/go): look up modules from a go-import mod proxy (#47044)
* feat(datasource/go): look up modules from a `go-import` `mod` proxy

When interacting with a Go module, a host can advertise that a module is
served by a Go module proxy, instead of a Version Control System (VCS),
by noting the following in the `go-import` HTML meta tag:

    <meta name="go-import" content="bsr.example.com/gen/go mod https://bsr.example.com/gen/go">

Previously, we only handled the `git` protocol, which would lead to
users integrating with a (private) Buf Schema Registry (BSR) not being
able to download generated Go modules.

When interacting with public Go modules on `buf.build`, these are served
successfully through `proxy.golang.org`, but a private BSR can't be
routed through the public proxy, and therefore fail to be looked up.

We should instead follow how `go` does it[0] and add support for
directly calling a `mod` proxy.

We leverage `getDirectReleases()`s' checks to make sure we don't
try and follow this flow when we're on a "known" host, i.e. if we're
pulling a module called `github.com/...` or a vanity domain package name
that resolves to `gitlab.com`.

The `?go-get=1` will keep the in-memory HTTP cache's result.

[0]: https://go.dev/ref/mod#serving-from-proxy

Co-Authored-By: Claude Opus 5.5 <jamie.tanna+claude-code@mend.io>

* fixup! feat(datasource/go): look up modules from a `go-import` `mod` proxy

fixup! feat(datasource/go): look up modules from a `go-import` `mod` proxy

Co-Authored-By: Claude Sonnet 5.5 <jamie.tanna+claude-code@mend.io>

* fixup! feat(datasource/go): look up modules from a `go-import` `mod` proxy

* fixup! feat(datasource/go): look up modules from a `go-import` `mod` proxy

---------

Co-authored-by: Claude Opus 5.5 <jamie.tanna+claude-code@mend.io>
2026-10-15 11:31:04 +00:00
Jamie Tanna 28ed2ee25e fix(datasource/go): treat modules with go host rule credentials as private (#47031)
When determining whether a Go module is public - which infers whether to
cache it - we had previously miscategorised authenticated traffic to Go
module proxies, as we only looked for `GOPROXY`, `GONOPROXY` and
`GOPRIVATE`.

Instead, we should treat a package as private if there is an
authenticated Host Rule that matches the Go module.

We can do this by checking for `hostType: go`, to catch only Go module
specific authentication, and to avoid catching i.e. auth to
`github.com`.

If any credentials (`token`, `password` or `httpsPrivateKey`) are found,
we treat this as credentialed.

Co-authored-by: Claude Opus 5.5 <jamie.tanna+claude-code@mend.io>
2026-10-14 11:17:39 +00:00
Michael Kriese 4dc30ea9bc fix(http): share concurrent identical cache provider requests (#46627)
* fix(http): share concurrent identical cache provider requests

Identical requests that use a cache provider and run at the same time now
share one in-flight request instead of each fetching from the server.
Callers that join an in-flight request get a deep copy of the response.

This replaces the per-URL mutex, which only covered the cache check and
was released before the fetch finished.

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* fix(http): reuse cache providers for npm and typst lookups

The npm and typst datasources now use one module-level cache provider, so
concurrent lookups of the same package share one request. The request key
is only hashed for GET and HEAD requests and includes `throwHttpErrors`.

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
Co-Authored-By: Claude Sonnet 5.5 <michael.kriese+claude-code@mend.io>

* fix(http): don't share requests that differ in response type

Add `responseType` and `responseBecomesConfig` to the key of shared
requests, and test that requests with different headers aren't shared
and that a shared response is copied for every caller.

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
Co-Authored-By: Claude Sonnet 5.5 <michael.kriese+claude-code@mend.io>

* fix(http): keep in-flight requests per Http class and copy binary bodies

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

---------

Co-authored-by: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
2026-10-13 15:38:10 +00:00
Michael Kriese 690da3581f fix(datasource/gitea-tags): return no digest for a missing repo or tag (#46984)
A 404 from the tag or commit request in the digest lookup was thrown as
an HTTP error and failed the whole dependency lookup. Return `null`
instead, as gitlab-tags does, so only the digest is missing. This also
applies to the Forgejo datasources, which share this implementation.

Co-authored-by: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
2026-10-13 08:00:00 +00:00
Aleksandr Mezin 911b681cc5 feat(datasource/rpm): support zstd compressed "primary" files (#46372)
* feat(datasource/rpm): support zstd compressed "primary" files

* Improve util tests

* Improve tests, remove unnecessary .destroy() call
2026-10-12 15:46:43 +00:00
Juan Carlos Blanco Delgado 476456b3c2 fix(metadata): strip git+ scheme prefix from source urls for changelog fetching (#45941)
* fix(metadata): strip git+ scheme prefix from source urls for changelog fetching

* fix(metadata): normalize git+ssh GitLab source urls

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: update URL handling to support additional github and gitlab formats

#45941

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Michael Kriese <michael.kriese@visualon.de>
2026-10-12 15:14:57 +00:00
choeffer 33fe1468d8 feat(datasource/devbox): update default registry URL to new search index (#46853)
* fix(datasource): add devbox new defaultRegistryUrl

https://github.com/jetify-com/devbox/releases/tag/0.18.4

* test(datasource): update devbox expected registry URL
2026-10-12 12:36:41 +00:00
Sebastian Poxhofer fdedac145a fix(datasource/gitlab-tags): don't cache failed digest lookups (#46904) 2026-10-12 08:22:36 +00:00
Sebastian Poxhofer a5516980ae fix(datasource/gitlab-tags): encode the ref of digest lookups (#46905) 2026-10-12 08:21:01 +00:00
Michael Kriese 182de2759f fix(datasource): cache the digest per ref (#46819)
Co-authored-by: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
2026-10-10 10:46:10 +00:00
Michael Kriese f64ec3975a fix(cache): build all cache keys with a shared helper (#46811)
* refactor(datasource): separate registry URLs in cache keys with a colon

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
Co-Authored-By: Claude Sonnet 5.5 <michael.kriese+claude-code@mend.io>

* fix(datasource/go): separate the remaining cache key parts with a colon

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* refactor(datasource): build cache keys with a shared colon-joining helper

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* refactor: build all cache keys with buildCacheKey

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* refactor(http): keep GET package cache keys as the bare URL

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* refactor(cache): extend cache key helper to memory cache keys

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* refactor(datasource/go): prefix the goproxy cache keys with their purpose

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

---------

Co-authored-by: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
2026-10-10 10:46:09 +00:00
Sebastian Poxhofer c0501194c0 feat(datasource/gitlab-packages): replace getJsonUnchecked with validated getJson (#44143)
* fix(datasource/gitlab-packages): replace getJsonUnchecked with validated getJson

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(datasource/gitlab-packages): use LooseArray and drop types.ts

Apply review feedback: tolerate malformed entries via LooseArray, remove
the now unused types.ts, use the fetchJson base helper and cover the
failed validation paths.

* fix(datasource/gitlab-packages): accept null for optional fields

Wrap the package schema in DeepNullish so a null conan_package_name
validates like an absent one.

* refactor(datasource/gitlab-packages): map releases in schema transform

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-10-08 12:14:07 +00:00
Sebastian Poxhofer 101cb08b4e feat(datasource): report the datasource which served a lookup (#46808)
* feat(datasource): report the datasource which served a lookup

A datasource which delegates a lookup to another one, such as the go datasource to github-tags, returns the delegate's id as effectiveDatasource. The lookup copies it onto the update result, so that templates, matchJsonata and logs can tell which path answered.

* Update lib/util/template/index.ts

* style(template): format the effectiveDatasource description
2026-10-07 14:57:25 +00:00
Sebastian Poxhofer 3312bc0c6a refactor(datasource): share git-refs and git-tags ref handling (#46189)
* refactor(datasource): share git-refs and git-tags ref handling

* test(datasource/git-refs): cover a remote without HEAD
2026-10-07 11:51:25 +00:00
Sebastian Poxhofer 42805afe66 refactor(datasource): share the GitLab URL helpers across the GitLab datasources (#46188) 2026-10-07 07:07:08 +00:00
Sebastian Poxhofer 290f5dcff3 refactor(datasource): normalise github getDigest signatures (#46187) 2026-10-07 07:07:07 +00:00
Sebastian Poxhofer f03acf6cd7 refactor(datasource): derive github-release-attachments from github-releases (#46186)
* refactor(datasource): derive github-release-attachments from github-releases

* refactor(datasource): keep the github-releases id readonly and inherit its timestamp note
2026-10-07 07:07:06 +00:00
Sebastian Poxhofer dc5352d4d8 refactor(datasource): share a base class across the git host *-tags datasources (#46185)
* refactor(datasource): share a base class across the git host *-tags datasources

* refactor(datasource): make the git-host-tags helpers protected
2026-10-07 07:07:05 +00:00
Sebastian Poxhofer f07b34947a refactor(datasource): add fetchJson() helpers to the Datasource base class (#46112) 2026-10-07 07:07:04 +00:00
Sebastian Poxhofer ee0e84f775 refactor(datasource): pass the http client through the Datasource constructor (#46108) 2026-10-07 07:07:03 +00:00
Michael Kriese 9d59ac5084 feat(datasource/crate): use host rule credentials for git index registries (#46574)
* feat(datasource/crate): use host rule credentials for git index registries

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* test(datasource/crate): cover host rule authentication for the crates.io git index

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* test(datasource/crate): state that the crates.io git index needs no allowCustomCrateGitRegistries

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* test(datasource/crate): drop the git auth helper assertion

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* test(datasource/crate): name the git authentication tests after what they assert

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

* feat(manager/cargo): accept crate host rules for git auth in artifact updates

Co-Authored-By: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>

---------

Co-authored-by: Claude Opus 5.5 <michael.kriese+claude-code@mend.io>
2026-10-07 06:44:40 +00:00
Niels Pardon 83d6dede6a refactor(datasource/conda): match prefix.dev by hostname (#46364)
The prefix.dev branch was selected by string-prefix matching the whole
registry URL, which misses forms that are the same host (`http://`, a
bare `https://prefix.dev`) while the deprecated `fast.prefix.dev` alias
needed a second literal alongside it.

Compare the parsed hostname instead, in a `common.ts` helper so the host
list lives in one place. No behaviour change for the URL shapes that
already matched.
2026-10-01 14:46:45 +00:00
Jamie Tanna 57bd81f3ac fix(datasource/go): check GitHub Tags for a later release timestamp (#46631)
Similar to changes in d8705054b0, we
should make sure that when there is an annotated Git tag (on GitHub) we
use the tag's `tagger.date` by using the `queryTags()`, and then using
the latest date, instead of relying on the GO module proxy's `Time`
field.

We can use an example from `github.com/kr/pretty@v0.3.1`, whose tag was
created some weeks after the commit it points to.

Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
2026-10-01 10:49:38 +00:00
Sebastian Poxhofer d706bb2e25 refactor(datasource): derive the lookup config types from each other (#46107)
`GetReleasesConfig`, `GetPkgReleasesConfig`, `GetDigestInputConfig` and
`DigestConfig` re-declared the same fields independently and the registry
glued two of them with an internal intersection, spreading the whole lookup
config into every `getReleases()` call. Introduce `RegistryUrlsConfig` for
the registry URL inputs, extend the per-registry types into the index-level
ones, and pass datasources an explicit projection so implementations cannot
depend on undeclared fields.
2026-09-30 16:50:41 +00:00
Sebastian Poxhofer 0b16fce833 refactor(datasource): resolve registry URLs with early returns (#46106)
* refactor(datasource): stop double-caching releases at the index level

15 datasources set `caching = true`, which makes the registry wrapper in
`lib/modules/datasource/index.ts` store every `getReleases` result under
`datasource-releases-<id>` for 15 minutes, while their own `cached()`
wrapper already stores the same result under `datasource-<id>` with a
longer TTL and stale-fallback. Drop `caching` from those 15; the four
that rely on the index cache alone (dart-version, flutter-version,
maven, pypi) keep it.

`caching` is documented as requiring the datasource to return the
`isPrivate` flag, and none of these 15 ever set it, so the index layer
was caching private results unconditionally. The remaining `cached()`
layer honours `cachePrivatePackages` as before.

Also fix two cache-key/registry defects found alongside:

- `galaxy-collection` declares `registryStrategy = 'hunt'` but keyed its
  cache on the package name only, so a `null` from the first hunted
  registry was replayed for the second one. The registry URL is now part
  of the key, which invalidates existing entries once.
- `typst` ignores `config.registryUrl` and always reads its default
  registry, so it now sets `customRegistrySupport = false`. A configured
  `registryUrls` for typst is now warned about and ignored instead of
  being silently dropped.

* Merge branch 'main' into secustor/refactor/datasource-remove-index-caching

* fix(datasource): document `first` as the default registry strategy

The `hunt` JSDoc claimed to be the default for an undefined
`registryStrategy`, but the `Datasource` base class sets `first`, so the
`?? 'hunt'` fallback in the registry was only reachable from a test double.
Make the field required on `DatasourceApi`, move the "default" note to
`first`, drop the fallback, and delete the five overrides that restated the
base default.

* refactor(datasource): make `postprocessRelease` an optional method

`getDigest` is already optional on `DatasourceApi` and detected with a
presence check, but `postprocessRelease` was required and satisfied by a
no-op in the base class that the caller had to recognise by prototype
identity. Declare it optional like `getDigest`, delete the no-op, and test
for the method instead.

* refactor(datasource): drop the function form of `defaultRegistryUrls`

The `() => string[]` form has had no production user since it was added in
#15856; only the registry spec doubles used it, and four datasources index
into the field as an array. Narrow the type to `string[]`, collapse the
registry URL resolution to one path, and rewrite the spec doubles as plain
arrays.
2026-09-30 16:50:40 +00:00
Sebastian Poxhofer 7c85eeb920 refactor(datasource): make postprocessRelease an optional method (#46105)
`getDigest` is already optional on `DatasourceApi` and detected with a
presence check, but `postprocessRelease` was required and satisfied by a
no-op in the base class that the caller had to recognise by prototype
identity. Declare it optional like `getDigest`, delete the no-op, and test
for the method instead.
2026-09-30 16:50:39 +00:00
Sebastian Poxhofer fc57966a6e fix(datasource): document first as the default registry strategy (#46104)
* fix(datasource): document `first` as the default registry strategy

The `hunt` JSDoc claimed to be the default for an undefined
`registryStrategy`, but the `Datasource` base class sets `first`, so the
`?? 'hunt'` fallback in the registry was only reachable from a test double.
Make the field required on `DatasourceApi`, move the "default" note to
`first`, drop the fallback, and delete the five overrides that restated the
base default.

* test(datasource): set registryStrategy on the object-literal datasource doubles

`registryStrategy` is required on `DatasourceApi` since this PR, and the two
object-literal doubles added with the GraalVM datasource (#40497) did not set
it, which failed the type check.
2026-09-30 16:50:38 +00:00
Jamie Tanna 8b3a066756 test: remove redundant hostRules.clear() calls (#46584)
As we're now managing them via a `afterEach` in `test/setup.ts`, we can
remove any entries in tests themselves.

We don't update overrides of an outer hook or cases where there are more
explicit `clear()`s.

Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
2026-09-30 11:22:18 +00:00
Luca Bruno c5d1b432d1 feat(datasource/crate): rename allowCustomCrateGitRegistries and relax sparse registries cloning (#45632)
* chore(options): rename allowCustomCrateGitRegistries entry

* feat(datasource/crate): relax sparse registries cloning
2026-09-29 07:52:10 +00:00
Sebastian Poxhofer a31f03dd7f refactor(workers): assemble the per-dependency lookup config in one place (#46155)
* refactor(workers): assemble the per-dependency lookup config in one place

The extracted constraints were merged with the same spread twice in
`fetch.ts`, once per package file and once per dependency, and
`constraintsVersioning` only at the package file level. Both levels now go
through `mergeExtractedConstraints()`, and `PackageDependency` gained the
`constraintsVersioning` which only `PackageFileContent` had.

The datasource `defaultConfig` was applied in `fetch.ts` before the lookup and
again in `flatten.ts` afterwards, because `Object.assign(dep, upd)` only copies
the `UpdateResult` fields back onto the dependency. Both call
`applyDatasourceDefaultConfig()` now. Persisting the merged defaults on the
dependency instead would have moved them into another precedence layer of
`flatten.ts`, which merges them over the update config, so the defaults keep
being applied twice.

Behaviour change: a `constraintsVersioning` set on a dependency by a manager is
now honoured, below the package file and the repository config. No manager sets
it yet.

* refactor(config): move applyDatasourceDefaultConfig next to getManagerConfig

Importing config/utils from the datasource module made tools/generate-imports.mjs load config options before manager-default-configs.generated.ts exists, breaking install in CI.

* Apply suggestion from @secustor

* refactor(config): drop any from applyDatasourceDefaultConfig
2026-09-29 07:38:07 +00:00
Jamie Tanna 6c26fc1c6a test(datasource): don't cache a HOST_BLOCKED result (#46473)
As a regression test to make sure this behaviour stays as it should be.

Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
2026-09-25 20:27:15 +00:00
Wes 027ef303fa fix(datasource/rubygems): preserve release timestamps for native gems (#46446)
fix(rubygems): preserve release timestamps for native gems

Include non-ruby platform suffixes when hashing release metadata so the identifiers match the RubyGems compact index. This prevents valid metadata from falling back to versions-only results and blocking minimum-release-age checks.

Cover generic, native, and absent platforms, cache reuse, mismatched platforms, and mixed-platform package lookups.
2026-09-25 14:35:53 +00:00
Sebastian Poxhofer a3d265d45d refactor(datasource): stop double-caching releases at the index level (#46103)
* refactor(datasource): stop double-caching releases at the index level

15 datasources set `caching = true`, which makes the registry wrapper in
`lib/modules/datasource/index.ts` store every `getReleases` result under
`datasource-releases-<id>` for 15 minutes, while their own `cached()`
wrapper already stores the same result under `datasource-<id>` with a
longer TTL and stale-fallback. Drop `caching` from those 15; the four
that rely on the index cache alone (dart-version, flutter-version,
maven, pypi) keep it.

`caching` is documented as requiring the datasource to return the
`isPrivate` flag, and none of these 15 ever set it, so the index layer
was caching private results unconditionally. The remaining `cached()`
layer honours `cachePrivatePackages` as before.

Also fix two cache-key/registry defects found alongside:

- `galaxy-collection` declares `registryStrategy = 'hunt'` but keyed its
  cache on the package name only, so a `null` from the first hunted
  registry was replayed for the second one. The registry URL is now part
  of the key, which invalidates existing entries once.
- `typst` ignores `config.registryUrl` and always reads its default
  registry, so it now sets `customRegistrySupport = false`. A configured
  `registryUrls` for typst is now warned about and ignored instead of
  being silently dropped.

* Merge branch 'main' into secustor/refactor/datasource-remove-index-caching
2026-09-24 21:07:52 +00:00
Michael Kriese e144d8dbc1 fix(http): validate the cached response shape in the http cache schema (#46417)
refactor(http): validate the cached response shape in the http cache schema

Co-authored-by: Claude Fable 5.1 <michael.kriese+claude-code@mend.io>
2026-09-23 14:12:16 +00:00
Sergei Zharinov ac2fc77641 refactor(http-mock): Remove request tracing (#44482)
* refactor(http-mock): remove request tracing

* test: replace remaining HTTP trace assertions
2026-09-23 09:01:01 +00:00
Gregor Zeitlinger 0d9888a4f2 feat(datasource): add GraalVM version datasource (#40497)
* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases

* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases

* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases

* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases

* feat(datasource): add GraalVM version datasource with support for JDK and JRE releases

* add coverage

* fix

* fix

* fix

* code cov

* code cov

* code cov

* refactor(datasource): integrate GraalVM into java-version datasource

Merges graalvm-version datasource into java-version as a unified multi-vendor
Java distribution datasource. Implements vendor-based modular architecture with
routing layer for Adoptium (Eclipse Temurin) and Oracle GraalVM.

Architecture:
- Vendor-specific handlers (adoptium.ts, graalvm.ts) with routing in index.ts
- Vendor-aware system detection with different arch/os mappings per vendor
- Custom registry support for GraalVM's mise-java API
- Adoptium remains the default registry for backwards compatibility

Package names:
- New explicit: adoptium-jdk, adoptium-jre, oracle-graalvm-jdk, oracle-graalvm-jre
- Legacy (map to Adoptium): java, java-jdk, java-jre

Changes:
- Created adoptium.ts and graalvm.ts vendor handlers
- Enhanced common.ts with vendor detection and vendor-specific system mappings
- Updated index.ts with routing layer and custom registry support
- Merged test suites with comprehensive coverage for both vendors
- Updated mise manager integration to use JavaVersionDatasource
- Removed graalvm-version datasource directory

All 65 datasource tests and 93 mise manager tests passing.

* fix

* test(datasource): add coverage for non-HttpError paths in java-version

* fix(datasource): remove dead code and add coverage for java-version

Remove unreachable HttpError handling from _getReleases catch block
since both getAdoptiumReleases and getGraalvmReleases handle HttpError
internally. Add test for the graalvmRegistryUrl fallback when
registryUrl is undefined.

* fix(datasource): address review comments for GraalVM datasource

- Remove unused 'datasource-graalvm-version' cache namespace
- Convert MiseJavaRelease interface to Zod schema and derive type from it
- Use http.getJson() with the Zod schema in getGraalvmReleases instead
  of getJsonUnchecked, letting the schema handle invalid/empty responses
- Update getGraalvmReleases test mock to use getJson

* docs: remove unused datasource-graalvm-version from cacheTtlOverride docs

* fix(datasource): preserve Adoptium config typing

* fix(datasource): harden GraalVM release parsing

* fix(datasource): address GraalVM review feedback

* test: cover GraalVM mise package mapping

* test(datasource): cover default GraalVM registry fallback

* test(datasource): cover legacy registry resolution

* test(datasource): type legacy datasource mocks

* fix(datasource): harden GraalVM defaults and responses

* refactor(datasource): use function-only registry accessors

* test(datasource): cover registry accessor functions

* test: restore coverage exclusions

* chore: retrigger CI checks
2026-09-23 08:40:24 +00:00
Sebastian Poxhofer 1e6ffa8171 refactor(datasource): add cached() helper to the Datasource base class (#46102)
Every datasource that caches its releases hand-wrote the same
`withCache({ namespace: `datasource-${id}`, ... }, () => this._getReleases(config))`
wrapper, repeating the namespace literal and using underscore-prefixed
method names that `docs/development/best-practices.md` discourages.

Add `Datasource.cacheNamespace` (derived from the datasource id) and a
protected `Datasource.cached()` helper that defaults `namespace` to it,
exporting `CachedOptions` from `with-cache.ts`. Migrate 29 datasources
onto the helper, dropping their namespace literals, the static
`cacheNamespace` field in `github-digest`, and renaming the wrapped
`_getX` methods to `fetchX`.

Behaviour change: `golang-version` used the constant cache key `'all'`
while declaring `customRegistrySupport = true`, so a custom registryUrl
shared the cache entry with the default one. The key is now
`${config.registryUrl}`, which invalidates existing entries once.
2026-09-22 19:30:25 +00:00
Sebastian Poxhofer a4b263a21e fix(datasource): declare metadata as instance fields, not statics (#46101)
Seven datasources (bitbucket-tags, bitbucket-server-tags,
gitlab-releases, gitea-tags, gitea-releases, forgejo-tags,
forgejo-releases) declared DatasourceApi metadata fields
(defaultRegistryUrls, releaseTimestampSupport/Note,
sourceUrlSupport/Note) as class `static readonly` instead of
instance fields. The contract (DatasourceApi, datasource.ts) reads
these off the instance, so the base class defaults silently applied
instead. Convert them to instance fields (`override readonly`, the
terraform-module pattern), keeping a static array only where a
static helper (or lib/modules/datasource/go/) still needs it, backed
by `override readonly defaultRegistryUrls = <Class>.defaultRegistryUrls`.
Delete the redundant `static readonly registryStrategy = 'first'`
(gitlab-releases, bitbucket-tags) and `static readonly
customRegistrySupport = true` (bitbucket-tags), which only matched
the base default by coincidence.

Replace the dead 'validates datasource' test in
lib/modules/datasource/index.spec.ts, which filtered out every
class-based datasource before validating (all 82 are class-based), with
a metadata contract test over getDatasources(): no listed metadata
name may be declared as a static-only property, releaseTimestampNote
requires releaseTimestampSupport, and sourceUrlNote requires
sourceUrlSupport !== 'none'.

Behaviour changes:
- These 7 datasources now report a real defaultRegistryUrls (or
  releaseTimestampSupport/sourceUrlSupport) to the registry
  (lib/modules/datasource/index.ts), so requests without an explicit
  registryUrl now go through the registry-URL resolution and
  getRegistryReleases package-cache path instead of calling
  getReleases directly. getDigestConfig now receives a resolved
  registryUrl instead of undefined for these datasources.
- Generated docs (tools/docs/datasources.ts,
  tools/docs/minimum-release-age.ts) will render the real
  "Release timestamp support" / "Source URL support" values and list
  these datasources under minimumReleaseAge support, instead of "No".
2026-09-22 19:30:24 +00:00
Aleksandr Mezin cbe339c5fd refactor(datasource/rpm): remove "gzip" from identifiers (#46264) 2026-09-21 09:21:36 +00:00
Sebastian Poxhofer bd86caa165 refactor(datasource): share the Gitea datasources with Forgejo (#46095)
* refactor(datasource): share the Gitea datasources with Forgejo

forgejo-tags and forgejo-releases were copies of gitea-tags and gitea-releases, and both -releases datasources re-implemented the digest lookup of the -tags ones. A GiteaDatasource base class now holds the shared implementation; the Forgejo datasources only declare their id, default registry URL, cache namespace and Http client. Results are only written to the shared package cache when they come from the public default instance.

* test(datasource): cover the default registry fallback

`GiteaDatasource.getRegistryUrl()` falls back to the instance's default
registry when the caller passes none, which the datasource dispatcher never
does because it resolves a registry URL first. Call `getTagCommit()` on the
instance instead, where the parameter is declared optional, once per default
registry so the Forgejo case also proves the base class reads the subclass's
`defaultRegistryUrls` rather than the Gitea static field.
2026-09-21 08:25:46 +00:00
Sebastian Poxhofer 8f5f238b8f fix(datasource/crate): skip the web API of registries that do not implement it (#46329)
* fix(datasource/crate): only query the crates.io web API for crates.io

Private sparse registries such as Artifactory or CodeArtifact advertise an
`api` URL in their `config.json` for publishing, but do not implement the
crates.io-only read endpoints `api/v1/crates/<name>` and
`api/v1/crates/<name>/<version>`. Since #41536 Renovate called both for
every registry with an `api` field, producing 404s and a
"Release interceptor failed" warning on each run.

Restore the crates.io flavor guard for the crate metadata lookup and the
release timestamp lookup. Reading `config.json` from a cloned registry is
now unreachable, so drop that branch.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(datasource/crate): cite the registry web API specification in the guards

* fix(datasource/crate): probe the registry web API once instead of guarding on flavor

Keep the crates.io read endpoints for registries that implement them.
Registries whose config.json points `api` at crates.io, such as index
mirrors, are used without probing. Any other registry is probed on the
first crate; a 404 for a crate that the index just returned means the
endpoint does not exist, so the registry is remembered as unsupported in
the memory cache and, for a day, in the package cache. The release
timestamp lookup handles a 404 the same way instead of surfacing the
generic "Release interceptor failed" warning.

Restores the cloned config.json branch, since cloned registries can now
advertise an API again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(datasource/crate): key the unsupported web API marker by the api URL

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-21 07:29:50 +00:00
Jamie Tanna 0253f4b23e fix(manager/dockerfile): don't attempt to update apk/apt for unknown image families (#46325)
If we don't discover the image family for a given image, instead of
suggesting a package update that may not exist - as we may look up a
`defaultRegistryUrls` entry that doesn't match the distro we're using -
we should not suggest any update.

In these cases, we can specify the `skipReason: unknown-registry`.

In #45706, we'll try and auto-detect this where possible.

Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
2026-09-18 15:59:58 +00:00
Sebastian Poxhofer 17466ceb32 fix(host-rules): ignore URL credentials when matching matchHost (#46128)
feat(host-rules): ignore URL credentials when matching matchHost

`matchesHost()` compared the raw href of the searched URL against
`matchHost`, so a URL carrying `user:pass@` userinfo (often a literal
placeholder such as `${USER}:${PASS}@`) never matched a `matchHost`
without credentials, including path-scoped ones. Callers worked around
this by stripping `username`/`password` off the URL by hand before
calling `hostRules.find()`.

- Add `stripUrlCredentials()` to `lib/util/url.ts`, returning the URL's
  `href` without userinfo (or `null` if unparseable), covered by
  `lib/util/url.spec.ts`.
- `matchesHost()` in `lib/util/host-rules.ts` now strips credentials
  from the searched URL before comparing it against an HTTP(S)
  `matchHost`, so `hostRules.find()` ignores userinfo on every caller's
  URL. Covered by new cases in `lib/util/host-rules.spec.ts` (a
  `matchHost` rule matches a URL carrying `user:pass@`, including a
  path-scoped `matchHost`).
- Remove the hand-rolled strip in
  `lib/modules/datasource/pypi/host-rules.ts`'s
  `findPypiIndexCredentials()`, now redundant since `find()` does it.
  `lib/modules/datasource/pypi/host-rules.spec.ts` already covers this
  behaviour and needed no changes.

This is strictly additive: more URLs match `matchHost` than before,
and nothing that matched before stops matching.
2026-09-18 10:53:12 +00:00