opening a url with urllib in python 3

Question

i'm trying to open the URL of this API from the sunlight foundation and return the data from the page in json. this is the code Ive produced, minus the parenthesis around myapikey.

import urllib.request.urlopen
import json

urllib.request.urlopen("https://sunlightlabs.github.io/congress/legislators?api_key='(myapikey)")

and im getting this error

Traceback (most recent call last):
  File "<input>", line 1, in <module>
ImportError: No module named request.urlopen

what am i doing wrong? ive researched into https://docs.python.org/3/library/urllib.request.html and still no progress

Answer 1

You need to use from urllib.request import urlopen, also I suggest you use the with statement while opening a connection.

from urllib.request import urlopen

with urlopen("https://sunlightlabs.github.io/congress/legislators?api_key='(myapikey)") as conn:
    # dosomething

Answer 2

In Python 3 You can implement that this way:

import urllib.request
u = urllib.request.urlopen("xxxx")#The url you want to open

Pay attention: Some IDE can import urllib(Spyder) directly, while some need to import urllib.request(PyCharm).

That's because you sometimes need to explicitly import the pieces you want, so the module doesn't need to load everything up when you just want a small part of it.

Hope this will help.

Answer 3

from urllib.request import urlopen
from bs4 import BeautifulSoup

wiki = "https://en.wikipedia.org/wiki/List_of_state_and_union_territory_capitals_in_India"

page = urlopen(wiki)
soup =  BeautifulSoup(page, "html.parser" ).encode('UTF-8')

print (soup)

Answer 4

urllib.request is a module whereas urlopen is a function. check out this link, it can help you clear your doubts. https://docs.python.org/3.0/library/urllib.request.html