mirror of
https://github.com/renovatebot/renovate.git
synced 2026-11-01 18:19:51 +00:00
Felipe Santos
2606ae2bd3
* feat(pypi): add support for JSON-based Simple API (PEP 691)
* Fix some stuff
* Clean some things up
* Fix coverage
* Simplify JSON parse
* Remove useless tests
* Align terminology
* Incorporate some changes from PR #44182
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
* Use isNonEmptyString to filter requires_python
* Address review feedback
* Warn once actually
* fixup! review comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! fixup! review comments
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! review comments
As the list of status codes we retry on was based on an unverified
claim that Artifactory, Nexus and devpi answer a rejected `Accept`
header with `400`/`403`/`415`, we should only retry on `406`, which is
what RFC 9110 specifies for a request whose `Accept` cannot be
satisfied.
Our `Accept` header ends with `text/html; q=0.01`, so any conforming
registry can satisfy it, `415` refers to the request body's
`Content-Type` rather than `Accept`, and retrying on `403` doubles the
requests for a registry we're not authenticated against.
The unwrapping of `ExternalHostError` is unaffected, as that's needed
for the `abortOnError` case regardless of which status codes we retry.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! review comments
As the note about `upload-time` sat in the paragraph about the Simple
API, it read as if `minimumReleaseAge` needs a registry which supports
the JSON-based Simple API (PEP 691).
That's only true of the Simple API. The JSON API has always exposed
`upload_time`, and as the default registry of `https://pypi.org/pypi/`
doesn't end in `/simple/`, that's the lookup most users get, so
`minimumReleaseAge` already works for them.
We can split the two apart, so that the guarantee for each API is
stated separately.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* reword
* fixup! review comments
As the JSON-based Simple API's release timestamps were only covered by
hand-written responses, we didn't have the equivalent of the real world
coverage added for the JSON API in
e942788539.
We can add a `numpy` fixture from PyPI's JSON-based Simple API, trimmed
to 1.5.0 and 1.5.1, so we assert against a response a registry has
actually served. 1.5.1 is the interesting case, as it was released in
2010 but gained a wheel in 2014 which PyPI lists first.
This also covers the `upload-time` format the Simple API uses, of
`2010-11-18T14:16:58.801332Z`, rather than the JSON API's
`2010-11-18T14:16:58`, so that we know `asTimestamp` normalises both.
Unlike the JSON API, there's no previous behaviour to characterise
first, as before these changes the Simple API exposed no timestamps at
all.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! fixup! review comments
* fixups
* fixups
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
---------
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <claude-code@jamietanna.co.uk>
15 lines
1.3 KiB
Markdown
15 lines
1.3 KiB
Markdown
This datasource uses the following logic to determine lookup URLs:
|
|
|
|
- If the normalized registryUrl ends in `/simple/` or `/+simple/` then only the simple API will be tried
|
|
- Otherwise, the JSON API will be tried first
|
|
- If the JSON API returns a result, it will be used
|
|
- If the JSON API throws an error (e.g. 403, 404) then the simple API will be tried
|
|
|
|
The Simple API supports both the HTML serialization ([PEP 503](https://peps.python.org/pep-0503/)) and the JSON serialization ([PEP 691](https://peps.python.org/pep-0691/)).
|
|
Renovate uses content negotiation (an `Accept` header) to prefer the JSON serialization, falling back to HTML automatically if the registry does not support it.
|
|
If the registry responds `406` because it cannot serve any of the negotiated types, Renovate retries the request without that header.
|
|
|
|
Renovate reads the release timestamp from the `upload_time` of a version's files, or the `upload-time` ([PEP 700](https://peps.python.org/pep-0700/)), which the Simple JSON API exposes, so [`minimumReleaseAge`](../../../configuration-options.md#minimumreleaseage) works against it.
|
|
|
|
If you are using Artifactory, you must [explicitly enable the generation of `upload-time` metadata server-side](https://jfrog.com/help/r/artifactory-enabling-time-based-package-filtering-for-pypi-repositories).
|