rvest error: “Error in class(out) <- ”XMLNodeSet“ : attempt to set an attribute on NULL”

问题

I'm trying to scrape a set of web pages with the new rvest package. It works for most of the web pages but when there are no tabular entries for a particular letter, an error is returned.

# install the packages you need, as appropriate
install.packages("devtools")
library(devtools)
install_github("hadley/rvest")
library(rvest)

This code works OK because there are entries for the letter E on the web page.

# works OK
url <- "https://www.propertytaxcard.com/ShopHillsborough/participants/alph/E"
pg <- html_session(url, user_agent("Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0"))
pg %>% html_nodes(".sponsor-info .bold") %>% html_text()

This doesn't work because there are no entries for the letter F on the web page. The error message is "Error in class(out) <- "XMLNodeSet" : attempt to set an attribute on NULL"

# yields error message
url <- "https://www.propertytaxcard.com/ShopHillsborough/participants/alph/F"
pg <- html_session(url, user_agent("Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0"))   
pg %>% html_nodes(".sponsor-info .bold") %>% html_text()    

Any suggestions. Thanks in advance.

回答1:

You could always wrap the pg…html_nodes…html_text in try and test for the class afterwards:

tmp <- try(pg %>% html_nodes(".sponsor-info .bold") %>% html_text(), silent=TRUE)

if (class(tmp) == "character") {
  print("do stuff")
} else {
  print("do other stuff")
}

EDIT: one other option is to use the boolean() XPath operator and do the test that way:

html_nodes_exist <- function(rvest_session, xpath) {

  xpathApply(content(rvest_session$response, as="parsed"), 
             sprintf("boolean(%s)", xpath))

}

pg %>% html_nodes_exist("//td[@class='sponsor-info']/span[@class='bold']")

which will return TRUE if those nodes exist and FALSE if they don't (that function needs to be generalized to be able to use session and ["HTMLInternalDocument" "HTMLInternalDocument" "XMLInternalDocument" "XMLAbstractDocument"] objects and work with both CSS selectors as well as XPath, but it's a way to avoid try.

来源：https://stackoverflow.com/questions/26702569/rvest-error-error-in-classout-xmlnodeset-attempt-to-set-an-attribute