selenium-webdriver

问题 I put together a crude scraper that scrapes prices/airlines from Expedia: # Start the Server rD <- rsDriver(browser = "phantomjs", verbose = FALSE) # Assign the client remDr <- rD$client # Establish a wait for an element remDr$setImplicitWaitTimeout(1000) # Navigate to Expedia.com appurl <- "https://www.expedia.com/Flights-Search?flight-type=on&starDate=04/30/2017&mode=search&trip=oneway&leg1=from:Denver,+Colorado,to:Oslo,+Norway,departure:04/30/2017TANYT&passengers=children:0,adults:1" remDr

问题 I am trying to create a webcrawler using Selenium, but I get this error when I try to create the webdriver object. selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome version must be between 70 and 73 (Driver info: chromedriver=2.45.615291 (ec3682e3c9061c10f26ea9e5cdcf3c53f3f74387),platform=Windows NT 6.1.7601 SP1 x86_64) I downloaded the latest version of chromedriver (2.45) which requires Chrome 70-73. My current Chrome version is 68.0.3440.106

问题 I am trying to create a webcrawler using Selenium, but I get this error when I try to create the webdriver object. selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome version must be between 70 and 73 (Driver info: chromedriver=2.45.615291 (ec3682e3c9061c10f26ea9e5cdcf3c53f3f74387),platform=Windows NT 6.1.7601 SP1 x86_64) I downloaded the latest version of chromedriver (2.45) which requires Chrome 70-73. My current Chrome version is 68.0.3440.106

问题 I have created a python selenium script that should navigate through a website and collect people profiles (https://www.shearman.com/people). The program won't loop through the pages to collect the links. I have used this which doesn't work; try: # this is navigate to next page driver.find_element_by_xpath('//div[@id="searchResultsSection"]/ul/li[12]').click() time.sleep(1) except NoSuchElementException: break The syntax behind the next button can be seen below; <a href="" onclick=

问题 I have created a python selenium script that should navigate through a website and collect people profiles (https://www.shearman.com/people). The program won't loop through the pages to collect the links. I have used this which doesn't work; try: # this is navigate to next page driver.find_element_by_xpath('//div[@id="searchResultsSection"]/ul/li[12]').click() time.sleep(1) except NoSuchElementException: break The syntax behind the next button can be seen below; <a href="" onclick=

问题 I noticed that facebook has some weird class names that look computer generated. What I don't know is if these classes are at least constant over time or they change in some time interval? Maybe someone who has experience with that can answer. Only thing I can see is that when I exit Chrome and open it again it is still the same, so at least they don't change every browser session. So I'd guess the best way to go about scraping facebook would be to use some elements in user interface and

问题 This query consists of 2 related questions. I need to wait for an element to be invisible before I go to next step, hence I tried to define a custom method as below: public void waitToDisappear(long timeOutInSeconds, WebElement element) { WebDriverWait wait = new WebDriverWait(driver, timeOutInSeconds); wait.ignoring(org.openqa.selenium.NoSuchElementException.class); wait.until(ExpectedConditions.invisibilityOf(element)); } When I call this method as common.waitToDisappear(5, <WebElement>); ,

问题 This query consists of 2 related questions. I need to wait for an element to be invisible before I go to next step, hence I tried to define a custom method as below: public void waitToDisappear(long timeOutInSeconds, WebElement element) { WebDriverWait wait = new WebDriverWait(driver, timeOutInSeconds); wait.ignoring(org.openqa.selenium.NoSuchElementException.class); wait.until(ExpectedConditions.invisibilityOf(element)); } When I call this method as common.waitToDisappear(5, <WebElement>); ,

问题 I want to separate my code in to smaller functions. But had an issue as driver was not available to all functions. So i declared it as a constant (or is there a better way of doing this ?) but in 3rd function it is failing on line : Select dropdown_finance_product = new Select(driver.findElement(By.xpath("//select[@id='ResultsNumber']"))); Here is the console message : Exception in thread "main" java.lang.NullPointerException at Scraping.scrapeit.fetch_urls(scrapeit.java:49) at Scraping

问题 As this answer by myself: How to download files headless in Selenium (Java) when download happens in new tab? If download button trigger download action in opened new tab, I will switch to new tab and send command(as below code) to download file. def enable_download_in_headless_chrome(self, driver, download_path): driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command') params = { 'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow