urrlib2.urlopen: “Name or service not known” persists when starting script without internet connection

Question

I have this simple minimal 'working' example below that opens a connection to google every two seconds. When I run this script when I have a working internet connection, I get the Success message, and when I then disconnect, I get the Fail message and when I reconnect again I get the Success again. So far, so good.

However, when I start the script when the internet is disconnected, I get the Fail messages, and when I connect later, I never get the Success message. I keep getting the error:

urlopen error [Errno -2] Name or service not known

What is going on?

import urllib2, time

while True:
    try:
        print('Trying')
        response = urllib2.urlopen('http://www.google.com')
        print('Success')
        time.sleep(2)
    except Exception, e:
        print('Fail ' + str(e))
        time.sleep(2)

Answer 1

This happens because the DNS name "www.google.com" cannot be resolved. If there is no internet connection the DNS server is probably not reachable to resolve this entry.

It seems I misread your question the first time. The behaviour you describe is, on Linux, a peculiarity of glibc. It only reads "/etc/resolv.conf" once, when loading. glibc can be forced to re-read "/etc/resolv.conf" via the res_init() function.

One solution would be to wrap the res_init() function and call it before calling getaddrinfo() (which is indirectly used by urllib2.urlopen().

You might try the following (still assuming you're using Linux):

import ctypes
libc = ctypes.cdll.LoadLibrary('libc.so.6')
res_init = libc.__res_init
# ...
res_init()
response = urllib2.urlopen('http://www.google.com')

This might of course be optimized by waiting until "/etc/resolv.conf" is modified before calling res_init().

Another solution would be to install e.g. nscd (name service cache daemon).

Answer 2

For me, it was a proxy problem. Running the following before import urllib.request helped

import os
os.environ['http_proxy']=''
response = urllib.request.urlopen('http://www.google.com')