rcurl

Currently changing user_agent by passing different strings to the html_session() method. Is there also a way to change your IP address on a timer when scraping a website? You can use a proxy (which changes your ip) via use_proxy as follows: html_session("you-url", use_proxy("proxy-ip", port)) For more details see: ?httr::use_proxy To check if it is working you can do the following: require(httr) content(GET("https://ifconfig.co/json"), "parsed") content(GET("https://ifconfig.co/json", use_proxy("138.201.63.123", 31288)), "parsed") The first call will return your IP. The second call should

How I would write the following curl POST method using R and the RCurl package? curl -k -u myusername:mypassword -d '{"text":"Hello World!","level":"Noob"}' -H "Content-Type: application/json" -H "Accept: application/json" "http://api.website/v1/access?" FINAL EDIT: After emailing the great Duncan Temple Lang I have a solution: library(RCurl) library(RJSONIO) postForm("http://api.website/v1/access?", .opts = list(postfields = toJSON(list(text = "Hello World!", level = "Noob")), httpheader = c('Content-Type' = 'application/json', Accept = 'application/json'), userpwd = "Username:Password", ssl

I am using Tor in combination with R and would like to change my IP for each new request. The code I have is as follows: library(RCurl) opts <- list(proxy="127.0.0.1", proxyport=8118) for (i in 1:10) { con <- socketConnection(host="127.0.0.1",port=9051) # DOES NOT WORK writeLines("signal newnym", con=con) # DOES NOT WORK ip <- getURL("http://ifconfig.me/ip", .opts = opts) print(ip) Sys.sleep(1) } I am able to connect via Tor, however the two lines marked as 'DOES NOT WORK' don't seem to get the proper signal across to Tor, so the IP stays the same. Regards! I had a similar problem, but managed

As an intern in an economic research team, I was given the task to find a way to automatically collect specific data on a real estate ad website, using R. I assume that the concerned packages are XML and RCurl , but my understanding of their work is very limited. Here is the main page of the website: http://www.leboncoin.fr/ventes_immobilieres/offres/nord_pas_de_calais/?f=a&th=1&zz=59000 Ideally, I'd like to construct my database so that each row corresponds to an ad. Here is the detail of an ad: http://www.leboncoin.fr/ventes_immobilieres/197284216.htm?ca=17_s My variables are: the price (

Is there any way to assign variable to a http GET request form using rcurl ? eg: getURL("https://testme.com/www//LoginService/login?login=xyz&password=<variable>") I need to pass the value of password as a variable. Regards ... npjc What about using paste0 ? library("RCurl") mypw <- "may1989" basereq <- "https://testme.com/www//LoginService/login?login=xyz&password=" fullreq <- paste0( basereq, mypw, "/" ) So the full request looks like: fullreq ## [1] "https://testme.com/www//LoginService/login?login=xyz&password=may1989/" which you can curl in: getURL(fullreq) Try httr: library(httr) GET(