Screen Scraping a Javascript based webpage in Python

Question

I am working on a screen scraping tool in Python. But, as I look through the source of the webpage, I noticed that most of the data is coming through Javascript.

An

Answer 1

I use webkit, which is the browser renderer behind Chrome and Safari. There are Python bindings to webkit through Qt.

And here is a full Python example to execute JavaScript and extract the final HTML.