rvest

问题 I'm trying to scrape an irregular table from Wikipedia using rvest. The table has cells that span multiple rows. The documentation for html_table clearly states that this is a limitation. I'm just wondering if there's a workaround. The table looks like this: My code: library(rvest) url <- "https://en.wikipedia.org/wiki/Arizona_League" parks <- url %>% read_html() %>% html_nodes(xpath='/html/body/div[3]/div[3]/div[4]/div/table[2]') %>% html_table(fill=TRUE) %>% # fill=FALSE yields the same

问题 I'm trying to automate a process that involves downloading .zip files from a couple of web pages and extracting the .csvs they contain. The challenge is that the .zip file names, and thus the link addresses, change weekly or annually, depending on the page. Is there a way to scrape the current link addresses from those pages so I can then feed those addresses to a function that downloads the files? One of the target pages is this one. The file I want to download is the second bullet under the

问题 I'm trying to automate a process that involves downloading .zip files from a couple of web pages and extracting the .csvs they contain. The challenge is that the .zip file names, and thus the link addresses, change weekly or annually, depending on the page. Is there a way to scrape the current link addresses from those pages so I can then feed those addresses to a function that downloads the files? One of the target pages is this one. The file I want to download is the second bullet under the

问题 I am trying to get financial data from morningstar.com; I want to get i.e. MSFT yearly revenue data. They are in a row <div> of a main <div> table. I followed some samples to get the main table: url <- "http://financials.morningstar.com/income-statement/is.html?t=MSFT&region=usa&culture=en-US" table <- url %>% read_html() %>% html_nodes(xpath='//*[@id="sfcontent"]/div[3]/div[3]') %>% html_table() but I get an empty list() . html_nodes itself returns a {xml_nodeset (0)} that I don't know how

问题 library(rvest) df <- data.frame(Links = c("Qmobile_Noir-M6", "Qmobile_Noir-A1", "Qmobile_Noir-E8")) for(i in 1:3) { webpage <- read_html(paste0("https://www.whatmobile.com.pk/", df$Links[i])) data <- webpage %>% html_nodes(".specs") %>% .[[1]] %>% html_table(fill = TRUE) } want to make loop works for all 3 values in df$Links but above code just download the last one, and downloaded data must also be identical with variables (may be a new column with variables name) 回答1: The problem is in how

问题 library(rvest) df <- data.frame(Links = c("Qmobile_Noir-M6", "Qmobile_Noir-A1", "Qmobile_Noir-E8")) for(i in 1:3) { webpage <- read_html(paste0("https://www.whatmobile.com.pk/", df$Links[i])) data <- webpage %>% html_nodes(".specs") %>% .[[1]] %>% html_table(fill = TRUE) } want to make loop works for all 3 values in df$Links but above code just download the last one, and downloaded data must also be identical with variables (may be a new column with variables name) 回答1: The problem is in how

问题 I want to scrape all the names from this page. With the result of one tibble of three columns. My code only works if all the data is there hence my error: Error: Tibble columns must have consistent lengths, only values of length one are recycled: * Length 20: Columns `huisarts`, `url` * Length 21: Column `praktijk` How can I let my code run but fill with Na 's in tibble if the data isn't there. My code for a pauzing robot later used in scraper function: pauzing_robot <- function (periods = c

问题 I am trying to scrape all details ( Type Of Traveller, Seat Type,Route,Date Flown, Seat Comfort, Cabin Staff Service, Food & Beverages, Inflight Entertainment,Ground Service,Wifi & Connectivity,Value For Money ) inclusive of the star rating from the airline quality webpage https://www.airlinequality.com/airline-reviews/emirates/ Not Working as expected my_url<- c("https://www.airlinequality.com/airline-reviews/emirates/") review <- function(url){ review<- read_html(url) %>% html_nodes("

问题 I would like to scrape information from a webpage. There is a login screen, and when I am logged in, I can access all kinds off pages from which I would like to scrape information (such as the last name of a player, the object .lastName ). I am using R and the packages rvest and httr . Somehow, the login seems to work, but I am clueless how to be redirected to the page I need to get the info from. The login form can be accessed on http://kickbase.sky.de/anmelden and the relevant pages have

问题 I'm somewhat new to scraping with R, but I'm getting an error message that I can't make sense of. My code: url <- "https://en.wikipedia.org/wiki/California_State_Legislature,_2017%E2%80%9318_session" leg <- read_html(url) testdata <- leg %>% html_nodes('table') %>% .[6] %>% html_table() To which I get the response: Error in out[j + k, ] : subscript out of bounds When I swap out html_table with html_text I don't get the error. Any idea what I'm doing wrong? Thanks! 回答1: Hope this helps!