mirror of
https://github.com/renovatebot/renovate.git
synced 2026-11-01 18:19:51 +00:00
2606ae2bd3
* feat(pypi): add support for JSON-based Simple API (PEP 691)
* Fix some stuff
* Clean some things up
* Fix coverage
* Simplify JSON parse
* Remove useless tests
* Align terminology
* Incorporate some changes from PR #44182
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
* Use isNonEmptyString to filter requires_python
* Address review feedback
* Warn once actually
* fixup! review comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! fixup! review comments
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! review comments
As the list of status codes we retry on was based on an unverified
claim that Artifactory, Nexus and devpi answer a rejected `Accept`
header with `400`/`403`/`415`, we should only retry on `406`, which is
what RFC 9110 specifies for a request whose `Accept` cannot be
satisfied.
Our `Accept` header ends with `text/html; q=0.01`, so any conforming
registry can satisfy it, `415` refers to the request body's
`Content-Type` rather than `Accept`, and retrying on `403` doubles the
requests for a registry we're not authenticated against.
The unwrapping of `ExternalHostError` is unaffected, as that's needed
for the `abortOnError` case regardless of which status codes we retry.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! review comments
As the note about `upload-time` sat in the paragraph about the Simple
API, it read as if `minimumReleaseAge` needs a registry which supports
the JSON-based Simple API (PEP 691).
That's only true of the Simple API. The JSON API has always exposed
`upload_time`, and as the default registry of `https://pypi.org/pypi/`
doesn't end in `/simple/`, that's the lookup most users get, so
`minimumReleaseAge` already works for them.
We can split the two apart, so that the guarantee for each API is
stated separately.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* reword
* fixup! review comments
As the JSON-based Simple API's release timestamps were only covered by
hand-written responses, we didn't have the equivalent of the real world
coverage added for the JSON API in
e942788539.
We can add a `numpy` fixture from PyPI's JSON-based Simple API, trimmed
to 1.5.0 and 1.5.1, so we assert against a response a registry has
actually served. 1.5.1 is the interesting case, as it was released in
2010 but gained a wheel in 2014 which PyPI lists first.
This also covers the `upload-time` format the Simple API uses, of
`2010-11-18T14:16:58.801332Z`, rather than the JSON API's
`2010-11-18T14:16:58`, so that we know `asTimestamp` normalises both.
Unlike the JSON API, there's no previous behaviour to characterise
first, as before these changes the Simple API exposed no timestamps at
all.
Co-Authored-By: Claude Opus 5 <claude-code@jamietanna.co.uk>
* fixup! fixup! review comments
* fixups
* fixups
Co-authored-by: Claude Opus 5 <jamie.tanna+claude-code@mend.io>
* fixup! comments
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
---------
Co-authored-by: Andreas Hippler <andreas.hippler@goto.com>
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Opus 5 <claude-code@jamietanna.co.uk>
62 lines
2.0 KiB
TypeScript
62 lines
2.0 KiB
TypeScript
import { isTruthy } from '@sindresorhus/is';
|
|
import { z } from 'zod/v4';
|
|
import { DeepNullish, LooseArray } from '../../../util/schema-utils/index.ts';
|
|
|
|
export const PypiRelease = DeepNullish(
|
|
z.object({
|
|
requires_python: z.string().optional(),
|
|
upload_time: z.string().optional(),
|
|
yanked: z.boolean().optional().default(false),
|
|
}),
|
|
);
|
|
|
|
export type PypiRelease = z.infer<typeof PypiRelease>;
|
|
|
|
export const PypiResponse = DeepNullish(
|
|
z.object({
|
|
info: z
|
|
.object({
|
|
name: z.string().optional(),
|
|
home_page: z.string().optional(),
|
|
project_urls: z.record(z.string(), z.string().optional()).optional(),
|
|
})
|
|
.optional(),
|
|
releases: z.record(z.string(), z.array(PypiRelease)).optional(),
|
|
}),
|
|
);
|
|
|
|
export type PypiResponse = z.infer<typeof PypiResponse>;
|
|
|
|
// JSON-based Simple API (PEP 691) schemas
|
|
// https://peps.python.org/pep-0691/
|
|
export const PypiSimpleFile = DeepNullish(
|
|
z.object({
|
|
filename: z.string(),
|
|
'requires-python': z.string().optional(),
|
|
yanked: z.union([z.boolean(), z.string()]).optional().default(false),
|
|
// `upload-time` is specified by PEP 700, which mandates an ISO 8601 string and lets a server omit the key even at `api-version` 1.1.
|
|
// A string which `asTimestamp` cannot parse downstream degrades to a missing timestamp; a value of any other type fails validation, and `LooseArray` then drops the file entry.
|
|
'upload-time': z.string().optional(),
|
|
}),
|
|
).transform(
|
|
({
|
|
filename,
|
|
'requires-python': requires_python,
|
|
yanked,
|
|
'upload-time': upload_time,
|
|
}) => ({
|
|
filename,
|
|
requires_python,
|
|
// Per the Simple API spec, `yanked` is yanked when truthy
|
|
// https://packaging.python.org/en/latest/specifications/simple-repository-api/#json-based-simple-api-for-python-package-indexes
|
|
yanked: isTruthy(yanked),
|
|
upload_time,
|
|
}),
|
|
);
|
|
export type PypiSimpleFile = z.infer<typeof PypiSimpleFile>;
|
|
|
|
export const PypiSimpleResponse = z.object({
|
|
files: LooseArray(PypiSimpleFile),
|
|
});
|
|
export type PypiSimpleResponse = z.infer<typeof PypiSimpleResponse>;
|