How to deal with a page which fails to load and continue testing in Watir-Webdriver

Question

I have looked for answer on other questions but I can't find one.

My problem is that I have a number of results that I need to test but my script keeps failing when I hit an Url that doesn't load a page. The following Url does not load. When this Url does not load I want to continue with my testing.

http://www.mycounciltax.org.uk/results?postcode=WC1N1DF&search=Search

I have tried to use:

begin
  Timeout::timeout(30) do

    //enter part that is hanging
  end
end

However the script just exits. Below is the full script that I am using. The script will time out and exit on....

browser.goto "http://www.mycounciltax.org.uk/results?postcode=WC1N1DF&search=Search"

Any ideas on how to allow the script to continue (bypass the testing of this Url) and move onto the next?

require "watir-webdriver"
browser = Watir::Browser.new :ff
browser.goto "http://www.mycounciltax.org.uk/results?postcode=cv5+6bz&search=Search"
content = browser.table.trs.collect {|tr| [tr[0].text, tr[1].text, tr[2].text]}
require 'win32ole'

application = WIN32OLE.new('Excel.Application')
application.visible = TRUE
workbook = application.Workbooks.Add();
worksheet = workbook.Worksheets(1);
worksheet.visible

row = 1; column = 0
content.each do |array|
  array.each do |element|
    worksheet.Cells(1,1).offset(row,column).value = element #.offset(row,column)
    column += 1
  end
  row += 1
  column = 0
end


browser.goto "http://www.mycounciltax.org.uk/results?postcode=WC1N1DP&search=Search"
if browser.table.exists?
  content = browser.table.trs.collect {|tr| [tr[0].text, tr[1].text, tr[2].text]}
  row = 1; column = 0
  content.each do |array|
    array.each do |element|
      worksheet.Cells(1,125).offset(row,column).value = element #.offset(row,column)
      column += 1
    end
    row += 1
    column = 0
  end
else
  content =0
end



browser.goto "http://www.mycounciltax.org.uk/results?postcode=WC1N1DF&search=Search"
if browser.table.exists?
  content = browser.table.trs.collect {|tr| [tr[0].text, tr[1].text, tr[2].text]}
  row = 1; column = 0
  content.each do |array|
    array.each do |element|
      worksheet.Cells(1,130).offset(row,column).value = element #.offset(row,column)
      column += 1
    end
   row += 1
   column = 0
  end
else
  content =0
end


browser.goto "http://www.mycounciltax.org.uk/results?postcode=WC1N1DP&search=Search"
if browser.table.exists?
  content = browser.table.trs.collect {|tr| [tr[0].text, tr[1].text, tr[2].text]}
  row = 1; column = 0
  content.each do |array|
    array.each do |element|
      worksheet.Cells(1,135).offset(row,column).value = element #.offset(row,column)
      column += 1
    end
    row += 1
    column = 0
  end
else
  content =0
end

Update

I meant to say that I would like to test whether each url loads before performing the if else (which outputs the table into excel). Could I do something along the lines of this

require "watir-webdriver"
browser = Watir::Browser.new :ff
browser.goto "http://www.mycounciltax.org.uk/results?postcode=cv5+6bz&search=Search"
content = browser.table.trs.collect {|tr| [tr[0].text, tr[1].text, tr[2].text]}
require 'win32ole'

application = WIN32OLE.new('Excel.Application')
application.visible = TRUE
workbook = application.Workbooks.Add();
worksheet = workbook.Worksheets(1);
worksheet.visible

row = 1; column = 0
content.each do |array|
  array.each do |element|
    worksheet.Cells(1,1).offset(row,column).value = element #.offset(row,column)
    column += 1
  end
  row += 1
  column = 0
end

begin
  browser.goto "http://www.mycounciltax.org.uk/results?postcode=WC1N1DP&search=Search"
  if browser.table.exists?
    content = browser.table.trs.collect {|tr| [tr[0].text, tr[1].text, tr[2].text]}
    row = 1; column = 0
    content.each do |array|
      array.each do |element|
        worksheet.Cells(1,125).offset(row,column).value = element #.offset(row,column)
        column += 1
      end
      row += 1
      column = 0
    end
  else
    content =0
  end
rescue => e
  puts "rescued #{e}"
end

begin
  browser.goto "http://www.mycounciltax.org.uk/results?postcode=WC1N1DF&search=Search"
    if browser.table.exists?
      content = browser.table.trs.collect {|tr| [tr[0].text, tr[1].text, tr[2].text]}
      row = 1; column = 0
      content.each do |array|
        array.each do |element|
          worksheet.Cells(1,130).offset(row,column).value = element #.offset(row,column)
          column += 1
        end
       row += 1
       column = 0
     end
  else
    content =0
  end
rescue => e
  puts "rescued #{e}"
end

begin   
  browser.goto "http://www.mycounciltax.org.uk/results?postcode=WC1N1DP&search=Search"
  if browser.table.exists?
    content = browser.table.trs.collect {|tr| [tr[0].text, tr[1].text, tr[2].text]}
    row = 1; column = 0
    content.each do |array|
      array.each do |element|
        worksheet.Cells(1,135).offset(row,column).value = element #.offset(row,column)
        column += 1
      end
      row += 1
      column = 0
    end
  else
     content =0
  end
rescue => e
    puts "rescued #{e}"
end

Answer 1

I am not sure how would you test a page that did not load, but try something like this:

begin
  browser.goto "http://www.mycounciltax.org.uk/results?postcode=WC1N1DF&search=Search"
rescue => e
  puts "rescued #{e}"
end

Answer 2

This question you are asking, and this sort of problem probably points to a much larger problem in how you are organizing and running scripts.

The first suggestion I would have in most cases would be to use an existing test framework, test/unit cucumber, fitness etc. All of these are designed to run tests as small atomic items which report a failure if anything goes wrong and then move to the next test. (opposed to most homebrewed scripts that are very often a giant long sequence which breaks if something goes amiss and cannot cope with any unexpected failure, or some kind of processing loop stepping through a file, and still have the same kinds of issues. I'm not sure if that would apply to you as you seem to be using watir for scraping more than testing as far as I can tell.

If you are rolling your own framework, then this is something you need to design into the system. It's called 'exception handling' and the basic format is as described by Zeljko in his answer. A quick google search will find you multiple tutorials on this aspect of the ruby language.

Your edited code above is closer to this, but is looking pretty repetitive to me. (although since it is currently showing as all flush left it's hard to make sense of. So I edited it to add indentation, ah that's better...) I would consider turning stuff you repeat three times into a method that takes in the URL and spreadsheet location as parameters which would cut way down on the repetition.

Also you might find it more useful to wrap just a small number of lines of code inside the begin/rescue/end in order to report a more useful message about what failed. That would allow you to report something like "error loading page #{url}" you could even write that out to your spreadsheet.

I'd recommend a little reading on how errors can 'bubble up' through the system so that failures can be a little more informative.