How can I get html content written by JavaScript with Selenium/Python [duplicate]

Answer 1

Since I usually do post-processing on the fetched nodes I run JavaScript directly in the browser with execute_script. For example to get all a-tags:

js_code = "return document.getElementsByTagName('a')"
your_elements = sel.execute_script(js_code)

Edit: execute_script and get_eval are equivalent except that get_eval performs an implicit return, in execute_script it has to be stated explicitly.

Answer 2

You would need to use a browser engine which can execute Javascript, such as PhantomJS. Javascript's changes are only visible to clients which can execute Javascript and provide a DOM/Runtime for events to be fired.

Also very close in relation to: Executing Javascript from Python

Answer 3

Can't you just call the browser object inside your selenium environment? For example:

self.browser.find_elements_by_tag_name("div")

Should return you an array of divs. You can also find by class, id, and so on.

Edit Below is the code to create your 'browser' object.

from selenium import webdriver #The browser object
self.browser = webdriver.Firefox() #I Use firefox, but can do chrome, IE, and safari i believe

Then you should be able to do as shown above with the find_elements_by_tag_name.