When I use WebDriverWait rather than Thread.sleep I get a StaleElementReferenceException

Question

I am trying to write a Selenium object Builder for a Swagger page, in groovy. For the purposes of this discussion, my problem code can be reduced down to the following:

class SwaggerBuilder {

    WebDriver driver
    def resources

    SwaggerBuilder(WebDriver driver) {

        this.driver = driver

        Thread.sleep(2000)

        resources = driver.findElements(By.className("resource")).collectEntries {
            def resourceName = it.findElement(By.tagName("a")).getText().replaceFirst("[/]", "")
            [(resourceName): it]
        }
    }

    Object invokeMethod(String name, Object args) {

        if(resources[(name)] == null)
            throw new NoSuchElementException("Resource $name cannot be found.")

        resources[(name)].findElement(By.linkText("List Operations")).click()
    }
}

Calling this is very simple (from JUnit3):

void test1() {

    driver = new FirefoxDriver()
    driver.get("http://petstore.swagger.wordnik.com/")

    def petstore = new SwaggerBuilder(driver)    // problem does not get past this line!
    try {
        petstore.not_there()
        fail("Did not catch exception.")
    } catch(NoSuchElementException ex) {
        assertTrue(ex.message.startsWith("Resource not_there cannot be found."))
    } catch(Exception ex) {
        fail("Caught wrong exception: ${ex.class}.")
    }
}

The Thread.sleep(2000) in the constructor is a horrible eye-sore! I tried to replace it with the following waits:

def wait = new WebDriverWait(driver, 20)
wait.until(ExpectedConditions.visibilityOfElementLocated(By.className("resource")))

or:

    def wait = new FluentWait<By>(By.className("resource")).
            withTimeout(20, TimeUnit.SECONDS).
            pollingEvery(100, TimeUnit.MILLISECONDS).
            ignoring(StaleElementReferenceException)
    wait.until(new Function<By, Boolean>() {
                def count = 0
                @Override
                Boolean apply(By by) {
                    def oldCount = count
                    count = driver.findElements(by).size()
                    return count == oldCount
                }
            })

Both of these produced the same result: "org.openqa.selenium.StaleElementReferenceException: Element is no longer attached to the DOM" in the closure on the line that starts with def resourceName = ....

The Thread.sleep(2000) is the only way that I can make this work right now. I am hoping to replace it with a more browser-friendly / robust wait, so this could work even for pages that load slower than 2 seconds. Any other ideas?

Answer 1

When I load your page, I see in the console "Loaded SwaggerUI" three times over. That's your problem: SwaggerUI is loaded 3 times over.

How to Figure This Out

So I did this:

1. I put a breakpoint over the line that prints out "Loaded SwaggerUI".

2. I reloaded.

3. When I hit the breakpoint, I took a snapshot of the elements that have the class resource:

   var snap1 = Array.prototype.slice.call(
                   document.getElementsByClassName("resource"))
   

   (You have to copy the returned value to an Array (with slice here) because getElementsByClassName returns a live collection.)

4. I hit the debugger's continue button.

5. When I hit the breakpoint again, I took a second snapshot (named snap2).

Ok now for some tests. If the DOM has not changed, the elements should be identical:

> snap1[0] === snap2[0]
false

That does not look good. Let's see what's still in the DOM tree:

> document.contains(snap1[0])
false
> document.contains(snap2[0])
true

The element in the first snapshot is no longer in the tree but the one in the 2nd is.

Why The Selenium Error?

The 2 second wait is enough to let Selenium start finding elements after the DOM has stabilized. However, when you tell Selenium to wait until there are visible elements of class resource in the page, it waits until SwaggerUI is loaded for the first time. At some point when it is processing the elements it first finds, SwaggerUI loads another time and then the old elements that it found are no longer in the DOM tree. So it raises a StaleElementReferenceException because the element it once found in the DOM tree is no longer there. It's been replaced by an element which is in the same location and structurally identical but Selenium wants the exact same element not an identical copy.

Answer 2

After the most excellent research / suggestion by @Louis, I ended up using:

def wait = new FluentWait<By>(By.className("resource")).
        withTimeout(10, TimeUnit.SECONDS).
        ignoring(NoSuchElementException)
wait.until(new Function<By, Boolean>() {
            WebElement res
            Boolean apply(By by) {
                def oldRes = res
                res = driver.findElement(by)
                return res == oldRes
            }
        })

If anyone is interested, the entire Builder can be found on SourceForge (still under construction at the time of this writing).