1. 技术文章
  2. Selenium Webdriver tests always fail on Azure DevOps but work perfectly on dev machine

Selenium Webdriver tests always fail on Azure DevOps but work perfectly on dev machine

由 这一生的挚爱 提交于 2020-04-17 17:54:23

问题


Using a .net core 3.1 XUnit test project and the C# bindings for the Selenium WebDriver API, my tests succeed when I run them locally on my development machine but always fail when run in an Azure DevOps release pipeline.

Here is the Yaml definition for the test task

steps:
- task: VSTest@2
  displayName: 'VsTest - testAssemblies'
  inputs:
    testAssemblyVer2: '**\--REDACTED--.dll'
    uiTests: true
    runInParallel: false
    rerunFailedTests: true

I have followed the microsoft guidelines and activated uiTest and ensured runInParallel is false in the pipeline definition. I have also tried running the tests with every possible permutation of these two

The ChromeDriver version is 80.0.3987.106, which is the version pre-installed on the agent I'm using. I target the ChromeDriver binaries pre-installed on the agent using the ChromeWebDriver environment variable (its value is C:\\SeleniumWebDrivers\\ChromeDriver). I have tried using the windows-2019 and the vs2017-win2016 agents. This did not change anything.

The web driver is running in headless mode.

Here is a relevant sample of the output from the Azure DevOps pipeline. I believe the test does not actually fail because an element could not be located (since the tests consistently work in local), rather because the driver itself fails.

[...]
2020-03-06T14:38:02.2076961Z Starting ChromeDriver 80.0.3987.106 (f68069574609230cf9b635cd784cfb1bf81bb53a-refs/branch-heads/3987@{#882}) on port 49976
2020-03-06T14:38:02.2078676Z Only local connections are allowed.
2020-03-06T14:38:02.2079642Z Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
2020-03-06T14:38:28.0289610Z [xUnit.net 00:00:31.64]       OpenQA.Selenium.NoSuchElementException : no such element: Unable to locate element: {"method":"css selector","selector":"#autocomplete"}
2020-03-06T14:38:28.0296747Z   (Session info: headless chrome=80.0.3987.122)
2020-03-06T14:38:28.0297355Z [xUnit.net 00:00:31.64]       Stack Trace:
2020-03-06T14:38:28.0328125Z ##[error][xUnit.net 00:00:31.64]     --REDACTED--Test.ArticleSearchTest.SearchJuraSaturdaySaturdayWithoutDatesAndNoOsmSelectionResultIsPageUfi [FAIL]
2020-03-06T14:38:28.0338184Z [xUnit.net 00:00:31.64]            at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
2020-03-06T14:38:28.0339005Z [xUnit.net 00:00:31.64]            at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
2020-03-06T14:38:28.0339833Z [xUnit.net 00:00:31.64]            at OpenQA.Selenium.Remote.RemoteWebDriver.FindElement(String mechanism, String value)
2020-03-06T14:38:28.0340530Z [xUnit.net 00:00:31.64]            at OpenQA.Selenium.Remote.RemoteWebDriver.FindElementById(String id)
2020-03-06T14:38:28.0341218Z [xUnit.net 00:00:31.64]            at OpenQA.Selenium.By.<>c__DisplayClass16_0.<Id>b__0(ISearchContext context)
2020-03-06T14:38:28.0341907Z [xUnit.net 00:00:31.64]            at OpenQA.Selenium.By.FindElement(ISearchContext context)
2020-03-06T14:38:28.0342538Z [xUnit.net 00:00:31.64]            at OpenQA.Selenium.Remote.RemoteWebDriver.FindElement(By by)
2020-03-06T14:38:28.0343418Z [xUnit.net 00:00:31.64]         C:\agent\_work\25\s\Tests\--REDACTED--Test\SeleniumSearchUtils.cs(16,0): at --REDACTED--Test.SeleniumSearchUtils.SetDestinationToJuraWithoutSelectingDestinationInDropdown(IWebDriver driver)
2020-03-06T14:38:28.0344566Z [xUnit.net 00:00:31.64]         C:\agent\_work\25\s\Tests\--REDACTED--Test\Tests\ArticleSearchTest.cs(118,0): at --REDACTED--Test.ArticleSearchTest.SearchJuraSaturdaySaturdayWithoutDatesAndNoOsmSelectionResultIsPageUfi()
2020-03-06T14:38:28.0345417Z [xUnit.net 00:00:31.65]       Output:
2020-03-06T14:38:28.0346024Z [xUnit.net 00:00:31.65]         Selenium target url: http://test.--REDACTED--.com/de/frankreich/camping-bretagne
2020-03-06T14:38:28.0375453Z [xUnit.net 00:00:31.65]   Finished:    --REDACTED--Test
[...]

As advised by Greg, I increased the allowed wait time before an element not found exception is thrown. I set it to 120 seconds, which lead to a timeout error Relevant stack trace

   ---- System.Net.WebException : The operation has timed out.
2020-03-06T15:39:37.4581244Z [xUnit.net 00:02:24.81]       Stack Trace:
2020-03-06T15:39:37.4582083Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.Remote.HttpCommandExecutor.MakeHttpRequest(HttpRequestInfo requestInfo)
2020-03-06T15:39:37.4582955Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.Remote.HttpCommandExecutor.Execute(Command commandToExecute)
2020-03-06T15:39:37.4583816Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.Remote.DriverServiceCommandExecutor.Execute(Command commandToExecute)
2020-03-06T15:39:37.4584863Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
2020-03-06T15:39:37.4585662Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.Remote.RemoteWebDriver.FindElement(String mechanism, String value)
2020-03-06T15:39:37.4586387Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.Remote.RemoteWebDriver.FindElementById(String id)
2020-03-06T15:39:37.4587146Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.By.<>c__DisplayClass16_0.<Id>b__0(ISearchContext context)
2020-03-06T15:39:37.4587838Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.By.FindElement(ISearchContext context)
2020-03-06T15:39:37.4588507Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.Remote.RemoteWebDriver.FindElement(By by)
2020-03-06T15:39:37.4589490Z [xUnit.net 00:02:24.81]         C:\agent\_work\25\s\Tests\--REDACTED--Test\SeleniumSearchUtils.cs(16,0): at --REDACTED--Test.SeleniumSearchUtils.SetDestinationToJuraWithoutSelectingDestinationInDropdown(IWebDriver driver)
2020-03-06T15:39:37.4590637Z [xUnit.net 00:02:24.81]         C:\agent\_work\25\s\Tests\--REDACTED--Test\Tests\ArticleSearchTest.cs(118,0): at --REDACTED--Test.ArticleSearchTest.SearchJuraSaturdaySaturdayWithoutDatesAndNoOsmSelectionResultIsPageUfi()
2020-03-06T15:39:37.4613895Z [xUnit.net 00:02:24.81]         ----- Inner Stack Trace -----
2020-03-06T15:39:37.4614741Z [xUnit.net 00:02:24.81]            at System.Net.HttpWebRequest.GetResponse()
2020-03-06T15:39:37.4615470Z [xUnit.net 00:02:24.81]            at OpenQA.Selenium.Remote.HttpCommandExecutor.MakeHttpRequest(HttpRequestInfo requestInfo)
2020-03-06T15:39:37.4616428Z [xUnit.net 00:02:24.81]       Output:
2020-03-06T15:39:37.4617060Z [xUnit.net 00:02:24.81]         Selenium target url: http://test.--REDACTED--.com/de/frankreich/camping-bretagne
2020-03-06T15:39:37.4617775Z [xUnit.net 00:02:24.82]   Finished:    --REDACTED--Test

Note: I tried adding the no-sandbox option to the ChromeDriver, to no avail. UPDATE: After going through the logs in details again, I stumbled upon this line:

OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server for URL http://localhost:49946/session/a0dac255bd41cc2bb8c6882761408f12/element timed out after 60 seconds.

I am absolutely positive the URL passed to the GoToUrl method is http://test.--REDACTED--.com/de/frankreich/camping-bretagne, valid URL (which is on a remote server, DNS resolution and all)

Could the localhost:49946 reference some kind of local selenium component that hasn't been launched properly ? I could not find any more information about this "back-end" in the error logs or on the internet ...


回答1:


You are getting a very explicit exception:

OpenQA.Selenium.NoSuchElementException : no such element: Unable to locate element: {"method":"css selector","selector":"#autocomplete"}

It is failing, because it cannot find the element. Since these tests work locally this leads me to believe the failure is due to a race condition between the web browser and Selenium. The common fix for this is to use a WebDriverWait object:

var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(30));
var element = wait.Until(ExpectedConditions.PresenceOfElementLocated(By.CssSelector("#autocomplete")));

// now element exists

These race conditions are common when you run tests on a fast local machine, but then those same tests fail on a build server, which is likely a lot busier than your machine.




回答2:


@GregBurghardt answer was near perfect (+1). A bit of more information in terms of:

  • The relevant HTML
  • Line of code
  • Binary version information

would have helped us to analyze the issue in a better way.

However, as you are seeing:

OpenQA.Selenium.NoSuchElementException : no such element: Unable to locate element: {"method":"css selector","selector":"#autocomplete"}

To locate/interact/click the element, you need to induce WebDriverWait in conjunction with ExpectedConditions as ElementToBeClickable() and you can use either of the following Locator Strategies:

  • Id:

    var element = new WebDriverWait(driver, TimeSpan.FromSeconds(20)).Until(ExpectedConditions.ElementToBeClickable(By.Id("autocomplete")));

  • CssSelector:

    var element = new WebDriverWait(driver, TimeSpan.FromSeconds(20)).Until(ExpectedConditions.ElementToBeClickable(By.CssSelector("#autocomplete")));

  • XPath:

    var element = new WebDriverWait(driver, TimeSpan.FromSeconds(20)).Until(ExpectedConditions.ElementToBeClickable(By.XPath("//*[@id='autocomplete']")));

Reference

You can find a couple of relevant discussions in:

  • Random Selenium E2e Tests Fail because of timeouts on Azure DevOps but work locally and with remote Selenium (BrowserStack Automate)
  • NoSuchElementException, Selenium unable to locate element


来源：https://stackoverflow.com/questions/60566905/selenium-webdriver-tests-always-fail-on-azure-devops-but-work-perfectly-on-dev-m

标签
selenium
selenium-webdriver
azure-devops
azure-pipelines
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!