Check if it's possible to scroll down with RSelenium

问题

I'm using RSelenium to automatically scroll down a social media website and save posts. Sometimes I get to the bottom of the webpage and no more posts can be loaded as no more data is available. I just want to be able to check if this is the case so I can stop trying to scroll.

How can I tell if it's possible to continue scrolling in RSelenium? The code below illustrates what I'm trying to do - I think I just need help with the "if" statement.

FYI there's a solution for doing this in Python here (essentially checking if page height changes between iterations), but I can't figure out how to implement it (or any another solution) in R.

# Open webpage
library(RSelenium)
rD = rsDriver(browser = "firefox")
remDr = rD[["client"]]
url = "https://stocktwits.com/symbol/NZDCHF"
remDr$navigate(url) 

# Keep scrolling down page, loading new content each time. 
ptm = proc.time()
repeat {   
  remDr$executeScript("window.scrollTo(0,document.body.scrollHeight);")
  Sys.sleep(3) #delay by 3sec to give chance to load. 

  # Here's where i need help  
  if([INSERT CONDITION TO CHECK IF SCROLL DOWN IS POSSIBLE]) {
    break
  }
}

回答1:

Stumbled across a way to do this in Python here and modified it to work in R. Below is a now-working update of the original code I posted above.

# Open webpage
library(RSelenium)
rD = rsDriver(browser = "firefox")
remDr = rD[["client"]]
url = "https://stocktwits.com/symbol/NZDCHF"
remDr$navigate(url) 

# Keep scrolling down page, loading new content each time. 
last_height = 0 #
repeat {   
  remDr$executeScript("window.scrollTo(0,document.body.scrollHeight);")
  Sys.sleep(3) #delay by 3sec to give chance to load. 

  # Updated if statement which breaks if we can't scroll further 
  new_height = remDr$executeScript("return document.body.scrollHeight")
  if(unlist(last_height) == unlist(new_height)) {
    break
  } else {
    last_height = new_height
  }
}

来源：https://stackoverflow.com/questions/45204934/check-if-its-possible-to-scroll-down-with-rselenium

标签

testing

automated-tests

rselenium