RSelenium fails to find element on page with dropdowns

问题

I'm trying to scrape some data from table that uses dropdowns. The table is, "GIC options". The "Term" is "Mid-Term" and the "Interest Type" is "Compound".

Looking at the SO answer here, I tried:

library(RSelenium)
remDr <- remoteDriver(port = 4445L)
remDr$navigate("https://www.nbc.ca/personal/savings-investments/gic/non-redeemable.html")
webElem <- remDr$findElement(using = "xpath", '//*[@class="ft2-dropdown-list-element"]/option[@value="Mid-Term"]')

and a css version

webElem <- remDr$findElement(using = "css", "#ft2-filterDropdownBtnID option[value='Mid-Term']")

Both result in Selenium message:Unable to locate element

How can I scrape the data in this table?

回答1:

Below my solution.

library(RSelenium)
driver <- rsDriver(browser=c("firefox"),port = 4445L)
remote_driver <- driver[["client"]]  
remote_driver$navigate("https://www.nbc.ca/personal/savings-investments/gic/non-redeemable.html")

#Active the box by a click
remote_driver$findElement(using = "css selector", '.ft2-dropdown-btn-label')$clickElement()

#Now you can choose what you need
remote_driver$findElement(using = "xpath", '//*[@id="ft2"]/div[2]/div/ul/li[3]/a')$clickElement()

#All
//*[@id="ft2"]/div[2]/div/ul/li[1]/a
#Short term
//*[@id="ft2"]/div[2]/div/ul/li[2]/a
#Mid term
//*[@id="ft2"]/div[2]/div/ul/li[3]/a
#Long term
//*[@id="ft2"]/div[2]/div/ul/li[4]/a

#To scrape the value of the table, below a possible solution.
webElem <- remote_driver$findElement(using = "css selector", 'div.table-wrapper:nth-child(4) > table:nth-child(1)')
webElem$getElementText()

来源：https://stackoverflow.com/questions/64830656/rselenium-fails-to-find-element-on-page-with-dropdowns

标签

rselenium