JSON API for PyPi - how to list packages?

Question

There is a JSON API for PyPI which allows getting data for packages:

http://pypi.python.org/pypi/<package_name>/json
http://pypi.python.org/pypi/<package_name>/<version>/json

However, is it possible to get a list of all PyPI packages (or, for example, recent ones) with a GET call?

Answer 1

The easiest way to do this is to use the simple index at PyPI which lists all packages without overhead. You can then request the JSON of each package individually by performing a GET request to the URLs mentioned in your question.

Answer 2

I know that you asked for a way to do this from the JSON API, but you can use the XML-RPC api to get this info very easily, without having to parse HTML.

try:
     import xmlrpclib
except ImportError:
     import xmlrpc.client as xmlrpclib

client = xmlrpclib.ServerProxy('https://pypi.python.org/pypi')
# get a list of package names
packages = client.list_packages()

Answer 3

I tried this answer, but it's not working on Python 3.6

I found one solution with HTML parsing by using lxml package, But you have to install it via pip command as

pip install lxml

Then, try the following snippet

from lxml import html
import requests

response = requests.get("https://pypi.org/simple/")

tree = html.fromstring(response.content)

package_list = [package for package in tree.xpath('//a/text()')]

Answer 4

Here's Bash one-liner:

curl -sG -H 'Host: pypi.org' -H 'Accept: application/json' https://pypi.org/pypi/numpy/json | awk -F "description\":\"" '{ print $2 }' |cut -d ',' -f 1

# NumPy is a general-purpose array-processing package designed to...