Find nth element by classname using selenium python

Question

I just started using selenium yesterday to help scrape some data and I'm having a difficult time wrapping my head around the selector engine. I know lxml, BeautifulSoup, jQuery and Sizzle have similar engines. But what I'm trying to do is:

1. Wait 10 seconds for page to completely load
2. Make sure there are the presence of ten or more span.eN elements (two load on intitial page load and more after)
3. Then start processing the data with beautifulsoup

I am struggling with the selenium conditions of either finding the nth element or locating the specific text that only exists in an nth element. I keep getting errors (timeout, NoSuchElement, etc)

    url = "http://someajaxiandomain.com/that-injects-html-after-pageload.aspx"
    wd = webdriver.Chrome()
    wd.implicitly_wait(10)
    wd.get(url)
    # what I've tried
    # .find_element_by_xpath("//span[@class='eN'][10]"))
    # .until(EC.text_to_be_present_in_element(By.CSS_SELECTOR, "css=span[class='eN']:contains('foo')"))

Answer 1

You need to understand the concept of Explicit Waits and Expected Conditions to wait for.

In your case, you can write a custom Expected Condition to wait for elements count found by a locator being equal to n:

from selenium.webdriver.support import expected_conditions as EC

class wait_for_n_elements_to_be_present(object):
    def __init__(self, locator, count):
        self.locator = locator
        self.count = count

    def __call__(self, driver):
        try:
            elements = EC._find_elements(driver, self.locator)
            return len(elements) >= self.count
        except StaleElementReferenceException:
            return False

Usage:

n = 10  # specify how many elements to wait for

wait = WebDriverWait(driver, 10)
wait.until(wait_for_n_elements_to_be_present((By.CSS_SELECTOR, 'span.eN'), n))

---

Probably, you could have also just used a built-in Expected Condition such as presence_of_element_located or visibility_of_element_located and wait for a single span.eN element to be present or visible, example:

wait = WebDriverWait(driver, 10)
wait.until(presence_of_element_located((By.CSS_SELECTOR, 'span.eN')))