mirror of
https://github.com/renovatebot/renovate.git
synced 2026-11-01 02:00:10 +00:00
Felipe Santos
2606ae2bd3
* feat(pypi): add support for JSON-based Simple API (PEP 691)
* Fix some stuff
* Clean some things up
* Fix coverage
* Simplify JSON parse
* Remove useless tests
* Align terminology
* Incorporate some changes from PR #44182
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
* Use isNonEmptyString to filter requires_python
* Address review feedback
* Warn once actually
* fixup! review comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! fixup! review comments
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! review comments
As the list of status codes we retry on was based on an unverified
claim that Artifactory, Nexus and devpi answer a rejected `Accept`
header with `400`/`403`/`415`, we should only retry on `406`, which is
what RFC 9110 specifies for a request whose `Accept` cannot be
satisfied.
Our `Accept` header ends with `text/html; q=0.01`, so any conforming
registry can satisfy it, `415` refers to the request body's
`Content-Type` rather than `Accept`, and retrying on `403` doubles the
requests for a registry we're not authenticated against.
The unwrapping of `ExternalHostError` is unaffected, as that's needed
for the `abortOnError` case regardless of which status codes we retry.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! review comments
As the note about `upload-time` sat in the paragraph about the Simple
API, it read as if `minimumReleaseAge` needs a registry which supports
the JSON-based Simple API (PEP 691).
That's only true of the Simple API. The JSON API has always exposed
`upload_time`, and as the default registry of `https://pypi.org/pypi/`
doesn't end in `/simple/`, that's the lookup most users get, so
`minimumReleaseAge` already works for them.
We can split the two apart, so that the guarantee for each API is
stated separately.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* reword
* fixup! review comments
As the JSON-based Simple API's release timestamps were only covered by
hand-written responses, we didn't have the equivalent of the real world
coverage added for the JSON API in
e942788539.
We can add a `numpy` fixture from PyPI's JSON-based Simple API, trimmed
to 1.5.0 and 1.5.1, so we assert against a response a registry has
actually served. 1.5.1 is the interesting case, as it was released in
2010 but gained a wheel in 2014 which PyPI lists first.
This also covers the `upload-time` format the Simple API uses, of
`2010-11-18T14:16:58.801332Z`, rather than the JSON API's
`2010-11-18T14:16:58`, so that we know `asTimestamp` normalises both.
Unlike the JSON API, there's no previous behaviour to characterise
first, as before these changes the Simple API exposed no timestamps at
all.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! fixup! review comments
* fixups
* fixups
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
---------
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <claude-code@jamietanna.co.uk>
1.3 KiB
1.3 KiB
This datasource uses the following logic to determine lookup URLs:
- If the normalized registryUrl ends in
/simple/or/+simple/then only the simple API will be tried - Otherwise, the JSON API will be tried first
- If the JSON API returns a result, it will be used
- If the JSON API throws an error (e.g. 403, 404) then the simple API will be tried
The Simple API supports both the HTML serialization (PEP 503) and the JSON serialization (PEP 691).
Renovate uses content negotiation (an Accept header) to prefer the JSON serialization, falling back to HTML automatically if the registry does not support it.
If the registry responds 406 because it cannot serve any of the negotiated types, Renovate retries the request without that header.
Renovate reads the release timestamp from the upload_time of a version's files, or the upload-time (PEP 700), which the Simple JSON API exposes, so minimumReleaseAge works against it.
If you are using Artifactory, you must explicitly enable the generation of upload-time metadata server-side.