问题
I am trying to scrape job titles from a dynamic job listing. When I use the function find_elements_by_class_name, the function doesn't return anything. I'm new to selenium so i'm not sure if i'm simply doing something incorrectly or misunderstanding the functionality.
The page im trying to scrape is: https://recruit.hirebridge.com/v3/CareerCenter/v2/?cid=7724
from selenium import webdriver
import time
#define the path for the chrome webdriver
chrome_path = r"C:/web/jobListing/chromedriver.exe"
#create a instance of the webdriver
driver = webdriver.Chrome(chrome_path)
driver.get("https://recruit.hirebridge.com/v3/CareerCenter/v2/?cid=7724")
time.sleep(10)
jobs = driver.find_elements_by_class_name("col-md-8 jobtitle")
print("starting print")
for job in jobs:
print(job.text)
回答1:
Root Cause:
col-md-8 and jobtitle are 2 different classes. When you use find_element_by_class_name it will internally convert the class name to a css selector and try to find the element.
Below is the evidence where find_element_by_class_name uses the css internally.
Solution:
As Selenium internally uses the css you have to make sure the classes are clubbed together meaning class1.class2.class3. In simple terms replace all white spaces with single dot in the class name from UI.
How to implement this to your situation:
You have to use the below syntax.
driver.find_element_by_class_name('col-md-8.jobtitle')
回答2:
Try:
jobs = driver.find_elements_by_xpath("//div[@class='col-md-8 jobtitle']/a")
I've switched the find element by class for xpath, this way you have more flexibility and it generally works better, I suggest you look into it!
回答3:
Seems like a bug? This works:
jobs = driver.execute_script("""
return document.getElementsByClassName("col-md-8 jobtitle")
""")
来源：https://stackoverflow.com/questions/58422998/selenium-python-find-elements-by-class-name-returns-nothing