Running xunit.net tests in VSTS

Question

I have an issue running xunit.net tests in VSTS. When running the Build plan, the Test assemblies step produces the following log:

2017-03-21T12:11:39.3302859Z ##[section]Starting: Test Assemblies
2017-03-21T12:11:39.3382932Z ==============================================================================
2017-03-21T12:11:39.3382932Z Task : Visual Studio Test
2017-03-21T12:11:39.3382932Z Description : Run tests with Visual Studio test runner
2017-03-21T12:11:39.3382932Z Version : 1.0.84
2017-03-21T12:11:39.3382932Z Author : Microsoft Corporation
2017-03-21T12:11:39.3382932Z Help : More Information
2017-03-21T12:11:39.3382932Z ==============================================================================
2017-03-21T12:11:39.3493151Z Preparing task execution handler.
2017-03-21T12:11:44.9245238Z Executing the powershell script: D:\a_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\1.0.84\VSTest.ps1
2017-03-21T12:11:46.6530959Z Testing container: 'REGISTRY::HKEY_CLASSES_ROOT\CLSID{177F0C4A-1CD3-4DE7-A32C-71DBBB9FA36D}'
2017-03-21T12:11:46.6530959Z
2017-03-21T12:11:46.6530959Z
2017-03-21T12:11:46.6810971Z Does not exist.
2017-03-21T12:11:46.6820975Z
2017-03-21T12:11:46.6820975Z
2017-03-21T12:11:46.8520939Z Working folder: D:\a\1\s
2017-03-21T12:11:46.8520939Z Executing C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "D:\a\1\s\Common.Tests\bin\Release\MyClassLibrary.Tests.dll" /Settings:"C:\Users\buildguest\AppData\Local\Temp\tmp89AE.tmp" /EnableCodeCoverage /logger:trx /TestAdapterPath:"D:\a\1\s"
2017-03-21T12:11:47.2730887Z Microsoft (R) Test Execution Command Line Tool Version 14.0.25420.1
2017-03-21T12:11:47.2740881Z Copyright (c) Microsoft Corporation. All rights reserved.
2017-03-21T12:11:47.2740881Z
2017-03-21T12:11:47.7430814Z Starting test execution, please wait...
2017-03-21T12:12:01.0768912Z Warning: [xUnit.net 00:00:01.1926376] Skipping: MyClassLibrary.Tests (Could not find any of the following assemblies: xunit.execution.desktop.dll)
2017-03-21T12:12:01.0768912Z
2017-03-21T12:12:01.1458970Z Warning: No test is available in D:\a\1\s\Common.Tests\bin\Release\MyClassLibrary.Tests.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.

The packages.config file for this project is this:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  ...
  <package id="xunit" version="2.2.0" targetFramework="net45" />
  <package id="xunit.abstractions" version="2.0.1" targetFramework="net45" />
  <package id="xunit.assert" version="2.2.0" targetFramework="net45" />
  <package id="xunit.core" version="2.2.0" targetFramework="net45" />
  <package id="xunit.extensibility.core" version="2.2.0" targetFramework="net45" />
  <package id="xunit.extensibility.execution" version="2.2.0" targetFramework="net45" />
  <package id="xunit.runner.msbuild" version="2.1.0" targetFramework="net45" />
  <package id="xunit.runner.visualstudio" version="2.2.0" targetFramework="net45" developmentDependency="true" />
</packages>

The values used in the Test assemblies build step:

Execution options

Test assembly: **\*Tests.dll;-:**\obj\**
Code coverage enabled: true

Advanced Execution options

VSTest: Version
VSTest version: Latest

Path to custom adapters is empty, but the tooltip states Nuget restored adapters are automatically searched for.

Reporting options

Platform: $(BuildPlatform)
Configuration: $(BuildConfiguration)

So it seems (to me) that it can find the proper file (MyClassLibrary.Tests.dll), but doesn't pick up the test methods (which are marked as [Fact] or [Theory]).

Anybody has an idea what I do wrong?

update

All projects in the solution are .NET 4.5

Answer 1

I had the same issue once I made the switch to xUnit, after lots of trial-and-error and searching I found that the xUnit test runner can only be run in an assembly targeting .NET 4.5.2 or above, otherwise it cannot be loaded (and thus the tests will not be found).

The documentation is a bit vague here, since the Getting Started with xUnit guide doesn't really mention this, it only mentions to create a new class library project "targeting .NET 4.5.2 (or later)".

But this is only for the assembly containing the actual tests, your projects being tested can still target .NET 4.5. In my case I simply switched my test assemblies to target .NET 4.5.2, but all the other projects still target .NET 4.5 and it works very well.

Answer 2

Just an update on this as I've recently also had this issue and couldn't get the VsTest runner to discover my .net Core 2.0 xunit tests.

Note: VsTest runner works fine locally on my machine, but not from VSTS - even though xunit test adapters are there.

In any case, use the .NET Core build task instead - it's pretty easy to use. Basically:

1. Add .Net Core build task (preview atm)
2. Select desired .NET Core version
3. Change command to test
4. Add path or paths to your test projects - e.g. **/*Tests.csproj
5. Any additional arguments - e.g --filter "Category!=Integration"
6. There's also a Publish test results option that'll create trx output file