rvest

I'm attempting to scrape data from http://www.footballoutsiders.com/stats/snapcounts , but I can't change the fields in the drop down boxes on the site ("team", "week", "position", and "year"). My attempt to scrape the table associated with team = "ALL", week= "1", pos = "All", and year= "2015" with rvest is below. url <- "http://www.footballoutsiders.com/stats/snapcounts" pgsession <- html_session(url) pgform <-html_form(pgsession)[[3]] filled_form <-set_values(pgform, "team" = "ALL", "week" = "1", "pos" = "ALL", "year" = "2015" ) submit_form(session=pgsession,form=filled_form, POST=url) y <-

I am using rvest to webscrape in R, and I'm running into memory issues. I have a 28,625 by 2 data frame of strings called urls that contains the links to the pages I'm scraping. A row of the frame contains two related links. I want to generate a 28,625 by 4 data frame Final with information scraped from the links. One piece of information is from the second link in a row, and the other three are from the first link. The xpaths to the three pieces of information are stored as strings in the vector xpaths . I am doing this with the following code: data <- rep("", 4 * 28625) k <- 1 for (i in 1

I am trying to simply replicate the example of rvest::html_nodes() , yet encounter an error: library(rvest) ateam <- read_html("http://www.boxofficemojo.com/movies/?id=ateam.htm") html_nodes(ateam, "center") Error in do.call(method, list(parsed_selector)) : could not find function "xpath_element" The same happens if I load packages such as httr , xml2 , selectr . I seem to have the latest version of these packages too... In which packages are functions such as xpath_element , xpath_combinedselector located? How do I get it to work? Note that I am running on Ubuntu 16.04, so that code might

Here's my code: library(rvest) #login url <- "https://secure.usnews.com/member/login?ref=https%3A%2F%2Fpremium.usnews.com%2Fbest-graduate-schools%2Ftop-medical-schools%2Fresearch-rankings" session <- html_session(url) form <- html_form(read_html(url))[[1]] filled_form <- set_values(form, username = "notmyrealemail", password = "notmyrealpassword") submit_form(session, filled_form) Here's what I get as output after submit_form : <session> https://premium.usnews.com/best-graduate-schools/top-medical-schools/research-rankings Status: 200 Type: text/html; charset=utf-8 Size: 286846 I assume this

I'm trying to scrape page below : https://metro.zakaz.ua/uk/?promotion=1 This page with react content. I can scrape first page with code: url="https://metro.zakaz.ua/uk/?promotion=1" read_html(url)%>% html_nodes("script")%>% .[[8]] %>% html_text()%>% fromJSON()%>% .$catalog%>%.$items%>% data.frame In result I have all items from first page, but I don't know how to scrape others pages. This js code move to other page if that can help: document.querySelectorAll('.catalog-pagination')[0].children[1].children[0].click() Thanks for any help! You will need 'RSelenum' to perform headless navigation.