There is a JSON API for PyPI which allows getting data for packages:
http://pypi.python.org/pypi//json
http://pypi.python.org/pypi/
I tried this answer, but it's not working on Python 3.6
I found one solution with HTML parsing by using lxml package, But you have to install it via pip command as
pip install lxml
Then, try the following snippet
from lxml import html
import requests
response = requests.get("https://pypi.org/simple/")
tree = html.fromstring(response.content)
package_list = [package for package in tree.xpath('//a/text()')]