Ruby Watir — Trying to loop through links in cnn.com and click each one of them

Question

I have created this method to loop through the links in a certain div in the web site. My porpose of the method Is to collect the links insert them in an array then click each one of them.

require 'watir-webdriver'
require 'watir-webdriver/wait'

site = Watir::Browser.new :chrome
url = "http://www.cnn.com/"
site.goto url

  box = Array.new
  container = site.div(class: "column zn__column--idx-1")
  wanted_links = container.links


  box << wanted_links
  wanted_links.each do |link|
    link.click
    site.goto url
    site.div(id: "nav__plain-header").wait_until_present
  end

site.close

So far it seems like I am only able to click on the first link then I get an error message stating this:

  unable to locate element, using {:element=>#<Selenium::WebDriver::Element:0x634e0a5400fdfade id="0.06177683611003881-3">} (Watir::Exception::UnknownObjectException)

I am very new to ruby. I appreciate any help. Thank you.

Answer 1

The problem is that once you navigate to another page, all of the element references (ie those in wanted_links) become stale. Even if you return to the same page, Watir/Selenium does not know it is the same page and does not know where the stored elements are.

If you are going to navigate away, you need to collect all of the data you need first. In this case, you just need the href values.

# Collect the href of each link
wanted_links = container.links.map(&:href)

# You have each page URL, so you can navigate directly without returning to the homepage
wanted_links.each do |link|
  site.goto url
end

In the event that the links do not directly navigate to a page (eg they execute JavaScript when clicked), you will need to collect enough data to re-locate the elements later. What you use as the locator will depend on what is known to be static/unique. As an example, I will assume that the link text is a good locator.

# Collect the text of each link
wanted_links = container.links.map(&:text)

# Iterate through the links
wanted_links.each do |link_text|
  container = site.div(class: "column zn__column--idx-1")
  container.link(text: link_text).click

  site.back
end