Which is the best programming language to write a web bot? [closed]

Question

I want know which programming language provides good number of libraries to program a web bot? Something like crawling a web page for data. Say I want fetch weather for weather.yahoo.com website.

Also will the answer be same for a AI desktop bot?

Answer 1

Here is how you could do it in Python:

from urllib2 import urlopen
from BeautifulSoup import BeautifulSoup
soup=BeautifulSoup(urlopen("http://weather.yahoo.com/").read())
for x in soup.find(attrs={"id":"myLocContainer"}).findAll("li"):
  print x.a["title"], x.em.contents

Prints:

Full forecast for Chicago, Illinois, United States (Haze) [u'35...47 °F']
Full forecast for London, Greater London, England (Light Rain) [u'43...45 °F']
Full forecast for New York, New York, United States (Partly Cloudy) [u'42...62 °F']
Full forecast for San Francisco, California, United States (Partly Cloudy) [u'51...70 °F']

Answer 2

I don't know if it is the best, but Python is definitely pretty good and simple for that.

Answer 3

Another good python library for screen scraping and web crawling is scrapy.