dotCover and xUnit not gathering coverage statistics in some environments

Question

We're using dotCover 2.7 and xUnit.net 1.9.2.

On my machine (Windows 7) and a co-worker's machine (Windows 8) we can run dotCover from the command line against one of our unit test assemblies that uses xUnit.net and get correct coverage reporting.

On our build machine (Windows Server 2008 R2 Standard) when we run the same thing the only code coverage that dotCover reports is the unit test assembly itself.

We're running xUnit.net using the MSBuild task. Here's the relevant pieces from the .msbuild file.

<UsingTask TaskName="Xunit.Runner.MSBuild.xunit" AssemblyFile="$(PackagesDir)xunit.$(XunitVersion)\lib\net20\xunit.runner.msbuild.dll" />

<Target Name="XunitTests">
    <xunit Assembly="$(TrunkDir)src.UnitTests\Ipseity.Server.Events.UnitTests\bin\Debug\Ipseity.Server.Events.UnitTests.dll" />
</Target>

We're running dotCover from the command line using the following command (same command in each environment).

"c:\Program Files (x86)\JetBrains\dotCover\v2.7\Bin\dotCover.exe" analyse /TargetExecutable="c:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild.exe" /TargetArguments="/t:XunitTests ipseity.msbuild" /TargetWorkingDir=c:\tfs\SI\ipseity\trunk\build /Output=c:\temp\coverage.xml

On the Windows 7 and Windows 8 machines the coverage.xml file contains the following.

<Root CoveredStatements="1977" TotalStatements="7867" CoveragePercent="25" ReportType="Xml" DotCoverVersion="2.7.2.176">
    <Assembly Name="Ipseity.Server.Common" CoveredStatements="4" TotalStatements="339" CoveragePercent="1">
        ...
    </Assembly>
    <Assembly Name="Ipseity.Server.Events" CoveredStatements="691" TotalStatements="798" CoveragePercent="87">
        ...
    </Assembly>
    <Assembly Name="Ipseity.Server.Events.UnitTests" CoveredStatements="1240" TotalStatements="1251" CoveragePercent="99">
        ...
    </Assembly>
    <Assembly Name="ipseity.Server.MessageProcessing" CoveredStatements="42" TotalStatements="5479" CoveragePercent="1">
        ...
    </Assembly>
</Root>

However on the build server (Windows Server 2008 R2 Standard) we only get the unit test assembly showing up in the coverage report.

<Root CoveredStatements="1033" TotalStatements="1039" CoveragePercent="99" ReportType="Xml" DotCoverVersion="2.7.2.176">
    <Assembly Name="Ipseity.Server.Events.UnitTests" CoveredStatements="1033" TotalStatements="1039" CoveragePercent="99">
        ...
    </Assembly>
</Root>

At this point we're baffled as to why we get different results on the build server than on our dev boxes so any suggestions as to what else to look for would be appreciated.

Answer 1

With the help of one of the DotCover developers we were finally able to figure out the problem.

DotCover requires that the PDB files be available during analysis. By default the MSBuild <xunit...> task shadow copies the assemblies into another folder to run the tests. This apparently does not copy the PDB files, only the assemblies.

So to fix the problem we simply had to turn shadow copying off.

Original MSBuild Task

<xunit Assembly="$(TrunkDir)\Ipseity.Server.Events.UnitTests.dll" />

Fixed MSBuild Task

<xunit Assembly="$(TrunkDir)\Ipseity.Server.Events.UnitTests.dll" ShadowCopy="False" />