How can I find multiple usernames that have the same class name in Python with Selenium?

问题

    if type == 2:
        for p in range(1, maxpages + 1):
            link = url + str(p)
            driver.get(link)

            Users = driver.find_elements_by_class_name("link span h5 hv7 secondary")
            print(Users[0].text)
            print(Users[1].text)
            print(Users[2].text)

I hope you can understand what I'm trying to do with the code above.

I'm trying to find ALL elements that have the class name: link span h5 hv7 secondary (there are over 100)

And then print them out each individually.

error for Nic's reply:

User: <selenium.webdriver.remote.webelement.WebElement (session="39c90e31145c4cc21937f591facb1d2d", element="880b3920-08ae-443b-a7ad-621ffd988ce8")>
Age: <selenium.webdriver.remote.webelement.WebElement (session="39c90e31145c4cc21937f591facb1d2d", element="dc0297b0-f21f-4492-8be4-e14b5126d2bb")>
Traceback (most recent call last):
  File "C:\Users\VENA2\PycharmProjects\pythonProject\main.py", line 85, in <module>
    print(f"Text: {user_container.find_element_by_css_selector('div.f6.nowrap.truncate')}")
  File "C:\Users\VENA2\PycharmProjects\pythonProject\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 430, in find_element_by_css_selector
    return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
  File "C:\Users\VENA2\PycharmProjects\pythonProject\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 658, in find_element
    return self._execute(Command.FIND_CHILD_ELEMENT,
  File "C:\Users\VENA2\PycharmProjects\pythonProject\venv\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute
    return self._parent.execute(command, params)
  File "C:\Users\VENA2\PycharmProjects\pythonProject\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\VENA2\PycharmProjects\pythonProject\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"div.f6.nowrap.truncate"}
  (Session info: chrome=87.0.4280.88)


Process finished with exit code 1

回答1:

Using find_elements_by_class_name you must replace each spaces by a "." as stated by @PDHide. This selected is limited to classes, I suggest to use css_selector with find_elements_by_css_selector. This allows you to mix ID, classNames, tagName..

The value you need to input is the same as a css query. First class must start with a "." and any classes on the same level must be separated by a "." as well. If their are multiple levels (child), separate with a space followed by ".".

More info on the css_selector with selenium here
More info on how to build your query value here

Once you have your query done, you may interact through the list using a for loop:

Users = driver.find_elements_by_css_selector("yourCssSelectorValue")
for user in Users:
    print(user.text)

From the screenshot provided, here a solution to get user/age/text.

First get the parent elements:

user_containers = driver.find_elements_by_css_selector('div.relative.flex-auto.mw-100')

Then get each Childs and print the results:

for user_container in user_containers:
    print(f"User: {user_container.find_element_by_css_selector('.link.span.h5.hv7.secondary').text}")
    print(f"Age: {user_container.find_element_by_css_selector('span.f6.fw7.silver').text}")
    print(f"Text: {user_container.find_element_by_css_selector('div.f6.nowrap.truncate').text}")

This is one way, but there will be a lot of read.

Another solution, you can also get all childs this way:

users = driver.find_element_by_css_selector('.link.span.h5.hv7.secondary')
age = driver.find_element_by_css_selector('span.f6.fw7.silver')
text = driver.find_element_by_css_selector('div.f6.nowrap.truncate')

Then zip the lists and print the result of each:

for values in zip(users, age, text):
    print(f"Users: {values[0].text}")
    print(f"Age: {values[1].text}")
    print(f"Text: {values[2].text}")

回答2:

space in class attribute indicates multiple classes so use '.' instead of space to indicate multiple classes:

   Users = driver.find_elements_by_class_name("link.span.h5.hv7.secondary")

You can use xpath or css to match exact class attribute value also:

CSS:

     Users = driver.find_elements_by_css_selector("[class='link span h5 hv7 secondary']")

Xpath:

     Users = driver.find_elements_by_xpath("//*[@class='link span h5 hv7 secondary']")

Example for using class with space:

from selenium import webdriver
from time import sleep

options = webdriver.ChromeOptions()
#options.headless = True
options.add_argument("--window-size=1920,1080")
options.add_argument("--headless")
options.add_argument("--disable-gpu")
options.add_argument(
    "user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36")
browser = webdriver.Chrome(options=options)
browser.get("https://www.instagram.com")
sleep(5)
#browser.refresh()
elem=browser.find_element_by_class_name('RP4i1.UVauz')
print(elem.get_attribute("outerHTML"))
browser.get_screenshot_as_file(f"screenshot.png")

Output:

<img class="RP4i1  UVauz" src="/static/images/homepage/screenshot1.jpg/d6bf0c928b5a.jpg" alt="">

回答3:

Just loop through them.

for user in Users:
    print(user.text)

来源：https://stackoverflow.com/questions/65444801/how-can-i-find-multiple-usernames-that-have-the-same-class-name-in-python-with-s

标签

python

selenium

xpath