automating the login to the uk data service website in R with RCurl or httr

Question

I am in the process of writing a collection of freely-downloadable R scripts for http://asdfree.com/ to help people analyze the complex sample survey data hosted by the UK d

Answer 1

I think one way to address "enter your organization" page goes like this:

library(tidyverse)
library(rvest)
library(stringr)

org <- "your_organization"
user <- "your_username"
password <- "your_password"

signin <- "http://esds.ac.uk/newRegistration/newLogin.asp"
handle_reset(signin)

# get to org page and enter org
p0 <- html_session(signin) %>% 
    follow_link("Login")
org_link <- html_nodes(p0, "option") %>% 
    str_subset(org) %>% 
    str_match('(?<=\\")[^"]*') %>%
    as.character()

f0 <- html_form(p0) %>%
    first() %>%
    set_values(origin = org_link)
fake_submit_button <- list(name = "submit-btn",
                           type = "submit",
                           value = "Continue",
                           checked = NULL,
                           disabled = NULL,
                           readonly = NULL,
                           required = FALSE)
attr(fake_submit_button, "class") <- "btn-enabled"
f0[["fields"]][["submit"]] <- fake_submit_button

c0 <- cookies(p0)$value
names(c0) <- cookies(p0)$name
p1 <- submit_form(session = p0, form = f0, config = set_cookies(.cookies = c0))

Unfortunately, that doesn't solve the whole problem—(2) is harder than it looks. I've got more of what I think is a solution posted here: R: use rvest (or httr) to log in to a site requiring cookies. Hopefully someone will help us get the rest of the way.