httr

I have a problem with logging in in my script. Despite all other good answers that I found on stackoverflow, none of the solutions worked for me. I am scraping a web forum for my PhD research, its URL is http://forum.axishistory.com . The webpage I want to scrape is the memberlist - a page that lists the links to all member profiles. One can only access the memberlist if logged in. If you try to access the memberlist without logging in, it shows you the log in form. The URL of the memberlist is this: http://forum.axishistory.com/memberlist.php . I tried the httr-package: library(httr) members

I am in the process of writing a collection of freely-downloadable R scripts for http://asdfree.com/ to help people analyze the complex sample survey data hosted by the UK data service . In addition to providing lots of statistics tutorials for these data sets, I also want to automate the download and importation of this survey data. In order to do that, I need to figure out how to programmatically log into this UK data service website . I have tried lots of different configurations of RCurl and httr to log in, but I'm making a mistake somewhere and I'm stuck. I have tried inspecting the

I need to download information from web site that is protected using cookies. I pass this protection manually and then insert cookies to httr . Here is similar topic, but it does not solve my problem: ( Copying cookie for httr ) library(httr) url<-"http://smida.gov.ua/db/emitent/year/xml/showform/32153/125/templ" cook<-"_SMIDA=9117a9eb136353bd6956651bd59acd37; __utmt=1; __utma=29983421.1729484844.1413489369.1413625619.1413627797.3; __utmb=29983421.7.10.1413627797; __utmc=29983421; __utmz=29983421.1413489369.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)" response <- GET(url, config(cookie=

I am trying to load some air pollution background data directly into R as a data.frame using the RCurl package. The website in question has 3 dropdown boxes to choose options before downloading the .csv file as shown in figure below: I am trying to select 3 values from the drop down box and download the data using "Download CSV" button directly into R as a data.frame. I want to download the different combinations of multiple years and multiple pollutants for a specific site. In other posts on StackOverflow I have come across getForm function from the RCurl package but I don't understand how to

i originally asked this question about performing this task with the httr package, but i don't think it's possible using httr . so i've re-written my code to use RCurl instead -- but i'm still tripping up on something probably related to the writefunction .. but i really don't understand why. you should be able to reproduce my work by using the 32-bit version of R, so you hit memory limits if you read anything into RAM. i need a solution that downloads directly to the hard disk. to start, this code to works -- the zipped file is appropriately saved to the disk. library(RCurl) filename <-

I'm trying to connect to the fitbit api using the httr library . Using the examples provided, I came up with the following code: library(httr) key <- '<edited>' secret <- '<edited>' tokenURL <- 'http://api.fitbit.com/oauth/request_token' accessTokenURL <- 'http://api.fitbit.com/oauth/access_token' authorizeURL <- 'https://www.fitbit.com/oauth/authorize' fbr <- oauth_app('fitbitR',key,secret) fitbit <- oauth_endpoint(tokenURL,authorizeURL,accessTokenURL) token <- oauth1.0_token(fitbit,fbr) sig <- sign_oauth1.0(fbr, token=token$oauth_token, token_secret=token$oauth_token_secret ) I get the

I am attempting to use this webpage http://volcano.si.edu/search_eruption.cfm to scrape data. There are two drop-down boxes that ask for filters of the data. I do not need filtered data, so I leave those blank and continue on to the next page by clicking " Search Eruptions ". What I have noticed, though, is that the resulting table only includes a small amount of columns (only 5) compared to the total amount of columns (total of 24) it should have. However, all 24 columns will be there if you click the " Download Results to Excel " button and open the downloaded file. This is what I need. So,