selenium

How to scrape a website thet has username and password?

你说的曾经没有我的故事 提交于 2021-02-11 12:49:43
问题 I am trying to scrape a website and make my program know all the buttons and links that inside of that website but my problem is that to get to the first page I need to enter a username and a password and then scraping the page that shows after that and every time it's scraping to the page with the password and the username someone knows how to do that? because I don't know-how this is the code that I tried: import requests import time from bs4 import BeautifulSoup from selenium import

Failed to navigate to https://www.google.ca. This usually means that a call to the COM method IWebBrowser2::Navigate2() with Python Unittest Selenium

一曲冷凌霜 提交于 2021-02-11 12:44:50
问题 from selenium import webdriver import unittest from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By from time import sleep class MyTestCase(unittest.TestCase): def setUp(self) -> None: self.driver = webdriver.Ie(executable_path="C:\\webdriver\\IEDriverServer.exe") self.driver.maximize_window() self.driver.get("https://www.google.ca") def test_googletest(self): element =

Iterate each user on instagram with specific conditions

烂漫一生 提交于 2021-02-11 12:42:40
问题 I am using selenium and im trying to filter a specific users on instagram with 2 condition : If the profile doesn't have a profile pic (instagram's default pic contains "YW5vbnl..." string) If i already follow this user enter image description here but he doesn't iterate all user, he just iterate 12 times the result of last user however this user is eligible Simple example: enter image description here enter image description here PS : sorry for mistake i am beginner on python 😁 try: browser

Iterate each user on instagram with specific conditions

旧时模样 提交于 2021-02-11 12:41:56
问题 I am using selenium and im trying to filter a specific users on instagram with 2 condition : If the profile doesn't have a profile pic (instagram's default pic contains "YW5vbnl..." string) If i already follow this user enter image description here but he doesn't iterate all user, he just iterate 12 times the result of last user however this user is eligible Simple example: enter image description here enter image description here PS : sorry for mistake i am beginner on python 😁 try: browser

Selenium IDE python code export, doesnt work

回眸只為那壹抹淺笑 提交于 2021-02-11 12:33:21
问题 I want to use Selenium IDE to do some easy tasks in Chrome an then export the code to python and execute there. However, when I execute the exported code in python nothing happens. # Generated by Selenium IDE import pytest import time import json from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.common.action_chains import ActionChains from selenium.webdriver.support import expected_conditions from selenium.webdriver.support.wait import

How to open MS Edge with specific profile with selenium webDriver?

最后都变了- 提交于 2021-02-11 12:32:23
问题 I am running automation on the edge browser. Edge browser supports profile and whenever i launch the edge from webdriver, it create new profile. Is there any way I can set the option to launch edge with given user profile ? 回答1: I use Java language as an example. You could use user-data-dir and profile-directory to use specific profile to launch Edge with Selenium. The sample code is like below: System.setProperty("webdriver.edge.driver", "your\\path\\to\\edge\\webdriver\\msedgedriver.exe");

Java code to stop seeing detailed logs in ChromeDriver?

不羁岁月 提交于 2021-02-11 12:32:23
问题 Can someone share a code which stops ChromeDriver to display logs in console. These red color logs are taking lots of space. Thanks 回答1: I hope the code below helps you to get rid of those red-coloured 'logs': System.setProperty("webdriver.chrome.silentOutput", "true"); java.util.logging.Logger.getLogger("org.openqa.selenium").setLevel(Level.SEVERE); 来源: https://stackoverflow.com/questions/61253792/java-code-to-stop-seeing-detailed-logs-in-chromedriver

Scraping with selenium and BeautifulSoup doesn´t return all the items in the page

六月ゝ 毕业季﹏ 提交于 2021-02-11 12:29:41
问题 So I came from the question here Now I am able to interact with the page, scroll down the page, close the popup that appears and click at the bottom to expand the page. The problem is when I count the items, the code only returns 20 and it should be 40. I have checked the code again and again - I'm missing something but I don't know what. See my code below: from selenium import webdriver from bs4 import BeautifulSoup import pandas as pd import time import datetime options = webdriver

Clicking multiple items on one page using selenium

戏子无情 提交于 2021-02-11 12:27:48
问题 My main purpose is to go to this specific website, to click each of the products, have enough time to scrape the data from the clicked product, then go back to click another product from the page until all the products are clicked through and scraped (The scraping code I have not included). My code opens up chrome to redirect to my desired website, generates a list of links to click by class_name. This is the part I am stuck on, I would believe I need a for-loop to iterate through the list of

How to click on a username within web.whatsapp.com using Selenium and Python

时间秒杀一切 提交于 2021-02-11 12:27:01
问题 The bot is supposed to send message on whatsApp WEB but unfortunately is stopping and giving error when asked find the user through X-path. from selenium import webdriver from selenium.webdriver.common.keys import Keys import time driver=webdriver.Chrome(executable_path="C:\drivers\chromedriver.exe") driver.get("https://web.whatsapp.com/") time.sleep(5) name= input("Enter name") input("Enter anything after scanning") time.sleep(2) user=driver.find_element_by_xpath("//span[@title='{}']".format