Issue Crawling Amazon, Element Cannot Be Scrolled into View

橙三吉。 提交于 2019-12-12 18:23:30

问题


I'm having an issue crawling pages on Amazon.

I've tried using:

  • Executing JS Script
  • Action Chains
  • Explicit Waits

Nothing seems to work. Everything throws one exception or error or another.

Base Script

ff = create_webdriver_instance()
ff.get('https://www.amazon.ca/gp/goldbox/ref=gbps_ftr_s-3_4bc8_dct_10-?gb_f_c2xvdC0z=sortOrder:BY_SCORE,discountRanges:10-25%252C25-50%252C50-70%252C70-&pf_rd_p=f5836aee-0969-4c39-9720-4f0cacf64bc8&pf_rd_s=slot-3&pf_rd_t=701&pf_rd_i=gb_main&pf_rd_m=A3DWYIK6Y9EEQB&pf_rd_r=CQ7KBNXT36G95190QJB1&ie=UTF8')
next_button = ff.find_element_by_xpath('(//li/a[contains(text(), "Next")])[1]')

Attempt #1: Executing JS

Script

ff.execute_script('arguments[0].scrollIntoView()', next_button)

Error

Element could not be scrolled into view

Attempt #2: Action Chain

Script

actions = ActionChains(ff)
actions.move_to_element(next_button)
actions.click(next_button)
actions.perform()

Error

TypeError: rect is undefined

Attempt #3: Explicit Wait

next_button = WebDriverWait(ff, 60).until(
    EC.visibility_of_element_located((By.XPATH, '(//li/a[contains(text(), "Next")])[1]'))
)

I've also tried using element_to_be_clickable. Both of these end up timing out.


回答1:


That's because you're trying to handle hidden link. Try below instead

next_button = ff.find_element_by_partial_link_text('Next')
next_button.click()

or

next _button = ff.find_element_by_link_text('Next→')

Note that find_element_by_partial_link_text/find_element_by_link_text searching for visible links only.

Also you might need to call

ff.implicitly_wait(10)

once in your script (somewhere after your WebDriver instance definition) or use ExplicitWait as below

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

next_button = WebDriverWait(ff, 10).until(EC.element_to_be_clickable((By.LINK_TEXT, 'Next→')))

to be sure that required element will be find even with rendering delay



来源:https://stackoverflow.com/questions/52356838/issue-crawling-amazon-element-cannot-be-scrolled-into-view

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!