Using a Ruby script to login to a website via https

Question

Alright, so here\'s the dealio: I\'m working on a Ruby app that\'ll take data from a website, and aggregate that data into an XML file.

The website I need to take data f

Answer 1

You can try use wget to fetch the page. You can analyse login process with this app www.portswigger.net/proxy/.

Answer 2

For what it's worth, you could check out Webrat. It is meant to be used a tool for automated acceptance tests, but I think you could use it to simulate filling out the login fields, then click through links by their names, and grab the needed HTML as a string. Haven't tried doing anything like it, tho.

Answer 3

Mechanize

Mechanize is ruby library which imititates the behaviour of a web browser. You can click links, fill out forms und submit them. It even has a history and remebers cookies. It seems your problem could be easily solved with the help of mechanize.

The following example is taken from http://docs.seattlerb.org/mechanize/EXAMPLES_rdoc.html:

require 'rubygems'
require 'mechanize'

a = Mechanize.new
a.get('http://rubyforge.org/') do |page|
  # Click the login link
  login_page = a.click(page.link_with(:text => /Log In/))

  # Submit the login form
  my_page = login_page.form_with(:action => '/account/login.php') do |f|
    f.form_loginname  = ARGV[0]
    f.form_pw         = ARGV[1]
  end.click_button

  my_page.links.each do |link|
    text = link.text.strip
    next unless text.length > 0
    puts text
  end
end