* feat(pypi): add support for JSON-based Simple API (PEP 691)
* Fix some stuff
* Clean some things up
* Fix coverage
* Simplify JSON parse
* Remove useless tests
* Align terminology
* Incorporate some changes from PR #44182
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
* Use isNonEmptyString to filter requires_python
* Address review feedback
* Warn once actually
* fixup! review comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! fixup! review comments
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! review comments
As the list of status codes we retry on was based on an unverified
claim that Artifactory, Nexus and devpi answer a rejected `Accept`
header with `400`/`403`/`415`, we should only retry on `406`, which is
what RFC 9110 specifies for a request whose `Accept` cannot be
satisfied.
Our `Accept` header ends with `text/html; q=0.01`, so any conforming
registry can satisfy it, `415` refers to the request body's
`Content-Type` rather than `Accept`, and retrying on `403` doubles the
requests for a registry we're not authenticated against.
The unwrapping of `ExternalHostError` is unaffected, as that's needed
for the `abortOnError` case regardless of which status codes we retry.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! review comments
As the note about `upload-time` sat in the paragraph about the Simple
API, it read as if `minimumReleaseAge` needs a registry which supports
the JSON-based Simple API (PEP 691).
That's only true of the Simple API. The JSON API has always exposed
`upload_time`, and as the default registry of `https://pypi.org/pypi/`
doesn't end in `/simple/`, that's the lookup most users get, so
`minimumReleaseAge` already works for them.
We can split the two apart, so that the guarantee for each API is
stated separately.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* reword
* fixup! review comments
As the JSON-based Simple API's release timestamps were only covered by
hand-written responses, we didn't have the equivalent of the real world
coverage added for the JSON API in
e942788539.
We can add a `numpy` fixture from PyPI's JSON-based Simple API, trimmed
to 1.5.0 and 1.5.1, so we assert against a response a registry has
actually served. 1.5.1 is the interesting case, as it was released in
2010 but gained a wheel in 2014 which PyPI lists first.
This also covers the `upload-time` format the Simple API uses, of
`2010-11-18T14:16:58.801332Z`, rather than the JSON API's
`2010-11-18T14:16:58`, so that we know `asTimestamp` normalises both.
Unlike the JSON API, there's no previous behaviour to characterise
first, as before these changes the Simple API exposed no timestamps at
all.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! fixup! review comments
* fixups
* fixups
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
---------
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <claude-code@jamietanna.co.uk>
Add regression tests for the fix that allows a null or missing
home_page in PyPI JSON responses (#43814). Covers both the schema
parser and the datasource, asserting that homepage is left unset and
sourceUrl/changelogUrl fall back to project_urls when home_page is null.
* fix(datasource): replace getJsonUnchecked with validated getJson in npm/Node/PyPI/Go
Replace all getJsonUnchecked calls in npm, Node.js, PyPI, and Go datasources
with Zod-validated getJson calls:
- datasource/npm: add NpmResponseSchema to schema.ts (shared with presets)
- config/presets/npm: migrate to NpmResponseSchema with passthrough version fields
- datasource/node-version: add NodeReleasesSchema using LooseArray
- datasource/pypi: add PypiJSONSchema (nullable+catch for lenient empty-body handling)
- datasource/go: add VersionInfoSchema for both versionInfo and getLatestVersion calls
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* some manual cleaning up
* removing schema level catch
* removing some more types
* refactor: rename schema
* fix linter errors
* remove schema suffix which hasn't been catched by linter
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Felipe Santos