Scraping websites with Javascript enabled?

Question

I\'m trying to scrape and submit information to websites that heavily rely on Javascript to do most of its actions. The website won\'t even work when i disable Javascript in

Answer 1

I've had exactly the same problem. It is not simple at all, but I finally found a great solution, using PyQt4.QtWebKit.

You will find the explanations on this webpage : http://blog.motane.lu/2009/07/07/downloading-a-pages-content-with-python-and-webkit/

I've tested it, I currently use it, and that's great !

Its great advantage is that it can run on a server, only using X, without a graphic environment.