R: rvest - is not proper UTF-8, indicate encoding?

Question

I'm trying out the "new" Rvest package from Hadley Wickham.

I've used it in the past, so I'd expected that everything run smoothly.

However, I keep seen this error:

> TV_Audio_Video_Marca <- read_html(page_source[[1]], encoding = "ISO-8859-1")
Error: Input is not proper UTF-8, indicate encoding !
Bytes: 0xCD 0x20 0x53 0x2E [9]

As you see in the code, I've use encoding: ISO-8859-1. Before that I was using "UTF-8", but function guess_encoding(page_source[[1]]) says that the encoding is: ISO-8859-1. I've tried with all the options provided by guess_encoding but none worked.

What is the problem?

My code:

library(RSelenium)
library(rvest)
#start RSelenium
checkForServer()
startServer()
remDr <- remoteDriver()
remDr$open()

#navigate to your page
remDr$navigate("http://www.linio.com.pe/tv-audio-y-video/televisores/")

#scroll down 5 times, waiting for the page to load at each time
for(i in 1:5){      
  remDr$executeScript(paste("scroll(0,",i*10000,");"))
  Sys.sleep(3)    
}

#get the page html
page_source<-remDr$getPageSource()

#parse it

TV_Audio_Video_Marca <- read_html(page_source[[1]], encoding = "UTF-16LE")

UPDATE 1

I've googled for "How to now the encoding of a web page?".

Found out this Makrup Validation Tool from W3C, but It wasn't of great help:

http://validator.w3.org/check?uri=http://www.w3.org/2003/10/empty/emptydoc.html

Answer 1

Looking at the page source, they claim to be using UTF-8 encoding:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

So, the question is, are they really using a different enough encoding we need to worry about, or can we just convert to utf-8, guessing that any errors will be negligible?

If you are happy with a quick and dirty approach, and some potential mojibake, you can just force utf-8 using iconv:

TV_Audio_Video_Marca <- read_html(iconv(page_source[[1]], to = "UTF-8"), encoding = "utf8")

In general, this is a bad idea - better to specify the encoding it's from. In this case, maybe the error is theirs, so this quick and dirty approach might be ok.