rselenium

RSelenium: Scraping a dynamically loaded page that loads slowly

蹲街弑〆低调 提交于 2021-02-18 18:39:50
问题 I'm not sure if it is because my internet is slow, but I'm trying to scrape a website that loads information as you scroll down the page. I'm executing a script that goes to the end of the page, and waits for the Selenium/Chrome server to load the additional content. The server does update and load the new content, because I am able to scrape information that wasn't on the page originally and the new content shows up on the chrome viewer, but it only updates once. I set a Sys.sleep() function

Using RSelenium to open chrome browser, getting “Unable to create new service: ChromeDriverService” message and error

℡╲_俬逩灬. 提交于 2021-02-11 14:11:21
问题 As the title says, I've got a script written using RSelenium to automate web navigation, using chrome as the browser. This script has worked fine for months now, but today when I tried to run it, I got the following message: Selenium message:Unable to create new service: ChromeDriverService Build info: version: '3.14.0', revision: 'aacccce0', time: '2018-08- 02T20:13:22.693Z' System info: host: 'LAPTOP-22QM8890', ip: '192.168.133.65', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0

Using RSelenium to open chrome browser, getting “Unable to create new service: ChromeDriverService” message and error

淺唱寂寞╮ 提交于 2021-02-11 14:10:37
问题 As the title says, I've got a script written using RSelenium to automate web navigation, using chrome as the browser. This script has worked fine for months now, but today when I tried to run it, I got the following message: Selenium message:Unable to create new service: ChromeDriverService Build info: version: '3.14.0', revision: 'aacccce0', time: '2018-08- 02T20:13:22.693Z' System info: host: 'LAPTOP-22QM8890', ip: '192.168.133.65', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0

How to zoom out page using RSelenium library in R?

痴心易碎 提交于 2021-02-08 20:38:29
问题 I am trying to write a web-scraper using RSelenium Library in R. The last step of my work includes taking screenshot of a table on web page. To fit the whole table into the window I should zoom out the web browser (in that case it's firefox). I tried to use: webElem <- remDR$findElement("css", "body") webElem$clickElement() webElem$sendKeysToElement(list(key = "control", "-")) however it doesn't work. I saw also this thread: Zoom out shiny app at default in browser and found there promising

Scraping data from LinkedIn using RSelenium (and rvest)

最后都变了- 提交于 2021-02-08 06:47:55
问题 I am trying to scrape some data from famous people on LinkedIn and I have a few problems. I would like do the following: On Hadley Wickhams page ( https://www.linkedin.com/in/hadleywickham/ ) I would like to use RSelenium to login and "click" the "Show 1 more education" - and also "Show 1 more experience" (note Hadley does not have the option to "Show 1 more experience" but does have the option to "Show 1 more education"). (by clicking the "Show more experience/education" allows me to scrape

Scraping data from LinkedIn using RSelenium (and rvest)

左心房为你撑大大i 提交于 2021-02-08 06:47:04
问题 I am trying to scrape some data from famous people on LinkedIn and I have a few problems. I would like do the following: On Hadley Wickhams page ( https://www.linkedin.com/in/hadleywickham/ ) I would like to use RSelenium to login and "click" the "Show 1 more education" - and also "Show 1 more experience" (note Hadley does not have the option to "Show 1 more experience" but does have the option to "Show 1 more education"). (by clicking the "Show more experience/education" allows me to scrape

Handle RSelenium error messages

半城伤御伤魂 提交于 2021-02-07 18:22:35
问题 Currently scraping a page which can have a variable amount of elements and also sometimes formats the same data elements with different selectors. Currently trying to ignore errors thrown by RSelenium with some tryCatch code but still stops when the specified element is not on the page: result <- tryCatch({ webElem <- remDr$findElement('xpath', "//tr[(((count(preceding-sibling::*) + 1) = 9) and parent::*)]//span[(((count(preceding-sibling::*) + 1) = 1) and parent::*)]") }, warning = function

How do I scrape information in this table using R?

心不动则不痛 提交于 2021-01-29 19:02:54
问题 I'm trying to scrape the following web page: https://www.timeanddate.com/weather/sweden/stockholm/historic?month=3&year=2020 I am interested in the table at the end, below "Stockholm Weather History for..." With the submitted code I am able to get the information for the 1st in the month, but I don't understand how to get it for the rest of the days. If I change the date in the drop-down list the url address doesn't change. How can I scrape that table for all days of the month? library

Rselenium / dialog box

北战南征 提交于 2021-01-29 15:15:21
问题 I am very new to RSelenium. I have a list of Finnish companies and I would like to extract the corresponding Business ID on the web page https://tietopalvelu.ytj.fi/yrityshaku.aspx?kielikoodi=3 A simple version of my R-code is the following: library(RSelenium) name_company <- c("nokia", "test") driver <- rsDriver(browser= 'firefox', port = 16L) remote_driver <- driver[["client"]] remote_driver$navigate("https://tietopalvelu.ytj.fi/yrityshaku.aspx?kielikoodi=3") input1 <- remote_driver

R: download pdf embedded in a webpage

痴心易碎 提交于 2021-01-29 04:14:47
问题 Trying to find an easier way to grab the table in the pdf embedded in this page or even better, download this pdf into a local drive: My code is below and results are messy... PS: none of the buttons at the bottom of the webpage works unless you use IE, and IE with RSelenium... I have created a code to load the page on IE, can successfully click on any of the buttons to either load the excel file (stuck at the step where it pops up a window to ask me to open or save) or open the pdf on the