selenium-webdriver

Selenium code is not able to scrape ofashion.com.cn

心不动则不痛 提交于 2020-04-11 18:12:06
问题 I was building a web scraper by using python selenium . The script scraped sites like amazon, stack overflow and flipcart but wasn't able to scrape ofashion. It is always returning me a blank .csv file. Here is my code: from selenium import webdriver from selenium.webdriver.chrome.options import Options import pandas as pd import time user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/537.36 (KHTML, like Gecko) ' \ 'Chrome/80.0.3987.132 Safari/537.36' driver_exe =

Selenium : Unable to put scraped elements to csv

夙愿已清 提交于 2020-04-11 18:01:33
问题 I have successfully scraped the data from the site. Well it's returning me error. I used "Title1" : pd.Series([ ele for ele.text in elements ]) for storing data to csv file, but returns me error that name "ele" not defined when i use element to .text method . When i remove .text , then it runs fine. But stores the id's which are not in form of text, so that's why i used .text . What is happening with usage of .text ? Here is my code : element = WebDriverWait(driver, 5).until( EC.presence_of

for a parent Element, there are two child Elements. If .text() of child 1 matches, how to perfomr action on Child2

时间秒杀一切 提交于 2020-04-11 17:59:07
问题 THe element is defined like this: <div class ="frm-group'> <label class ="ng.binding" "Source Environment" ..... </label> <select class "ng-touched" ...... </select> </div> <div class ="frm-group'> <label class ="ng.binding" "Target Environment" ..... </label> <select class "ng-touched" ...... </select> </div> Div class is the parent class and "lable" and "select" class is the child class. Problem Statement : I have to do the following :: If the Label.getText() matches with a string, then

How to rotate Selenium webrowser IP address

﹥>﹥吖頭↗ 提交于 2020-04-07 04:53:07
问题 I have a Python script that visits a website every 30 sec, and I would need to have a different IP address each time. What would be the best/most time effective solution? scraping free proxies online? Do you know a python script that gather proxies from many sources? use Tor browser to have a different IP each time (I'm using selenium on an aws ec2 instance, you guys know a tutorial on how to use Tor browser on Ubuntu server?) other methods? 回答1: To gather and use different proxies a robust

Selenium - Is it okay to mix implicit wait and explicit wait like this?

柔情痞子 提交于 2020-04-07 04:41:53
问题 Here is a udemy course (from "Lets Kode It") to develop a web automation framework with selenium and Java. But, this is not a java question. You only need to know selenium in any of these languages - javascript, python, ruby, c# & java. The instructor has developed a CustomDriver class which has the method/function given below. The method waits for an element to be clickable, without having to write WebDriverWait statements everywhere in our code. It first sets the implicit wait to zero, does

Selenium - Is it okay to mix implicit wait and explicit wait like this?

不羁的心 提交于 2020-04-07 04:41:27
问题 Here is a udemy course (from "Lets Kode It") to develop a web automation framework with selenium and Java. But, this is not a java question. You only need to know selenium in any of these languages - javascript, python, ruby, c# & java. The instructor has developed a CustomDriver class which has the method/function given below. The method waits for an element to be clickable, without having to write WebDriverWait statements everywhere in our code. It first sets the implicit wait to zero, does

ElementNotVisibleException: Message: element not interactable error while trying to click on the top video in a youtube search

时光怂恿深爱的人放手 提交于 2020-04-07 04:31:30
问题 I cannot seem to find a way to click on the right element in order to get the url I am looking for. In essence I am trying to click on the top video in a youtube search (the most highly ranked returned video). How to resolve ElementNotInteractableException: Element is not visible in Selenium webdriver? -> This is for Java but it let me in the right direction (knowing I needed to execute JavaScript) http://www.teachmeselenium.com/2018/04/17/python-selenium-interacting-with-the-browser

How to address the log INFO Using `new FirefoxOptions()` is preferred to `DesiredCapabilities.firefox()` in selenium project

人盡茶涼 提交于 2020-04-07 03:33:41
问题 I just started a selenium project, but things didn't got right, so after searching a bit I found this solution. It works but i can't understand what these red statements want me to do, or how to get rid of them. import org.openqa.selenium.WebDriver; import org.openqa.selenium.firefox.FirefoxBinary; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.firefox.FirefoxOptions; import org.openqa.selenium.remote.DesiredCapabilities; import java.io.*; public class SelTest1 {

How to address the log INFO Using `new FirefoxOptions()` is preferred to `DesiredCapabilities.firefox()` in selenium project

倾然丶 夕夏残阳落幕 提交于 2020-04-07 03:33:18
问题 I just started a selenium project, but things didn't got right, so after searching a bit I found this solution. It works but i can't understand what these red statements want me to do, or how to get rid of them. import org.openqa.selenium.WebDriver; import org.openqa.selenium.firefox.FirefoxBinary; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.firefox.FirefoxOptions; import org.openqa.selenium.remote.DesiredCapabilities; import java.io.*; public class SelTest1 {

How can I save test case result in a variable whether it is passed or not after the test case execution is done?

落爺英雄遲暮 提交于 2020-04-06 17:38:40
问题 I am using selenium webdriver in visual studio Using NUNIT C#. The code of test case is public class MainProjectFile { private IWebDriver WDriver; private log4net.ILog Testlog; [TestInitialize] public void wd() { WDriver = Helper.GetWebDriver(helperconst.browserType.Chrome); Testlog = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType); } [Priority(1)] [TestMethod] public void search() { Testlog.Info("Test ID:001, This test will select the criteria and update