Python check if website exists

Question

I wanted to check if a certain website exists, this is what I\'m doing:

user_agent = \'Mozilla/20.0.1 (compatible; MSIE 5.5; Windows NT)\'
headers = { \'User         


        

Answer 1

You can use HEAD request instead of GET. It will only download the header, but not the content. Then you can check the response status from the headers.

import httplib
c = httplib.HTTPConnection('www.example.com')
c.request("HEAD", '')
if c.getresponse().status == 200:
   print('web site exists')

or you can use urllib2

import urllib2
try:
    urllib2.urlopen('http://www.example.com/some_page')
except urllib2.HTTPError, e:
    print(e.code)
except urllib2.URLError, e:
    print(e.args)

or you can use requests

import requests
request = requests.get('http://www.example.com')
if request.status_code == 200:
    print('Web site exists')
else:
    print('Web site does not exist')