specflow fails when trying to generate test execution report

Question

I've got a project that's using SpecFlow, NUnit and Coypu to do acceptance tests on a web application. I've got the project building OK via Jenkins on a build server. Jenkins calls a psake script which runs msbuild on the specs project, then the script calls nunit-console to run the specs/tests, and then I want to generate a report from SpecFlow.

Framework "4.0"

task Default -depends RunSpecs

task BuildSpecs {
    $env:EnableNuGetPackageRestore = "true"
    msbuild /t:Rebuild ReturnsPortal.Specs.csproj
}

task RunSpecs -depends BuildSpecs {
    exec { & "C:\path\to\NUnit 2.5.9\bin\net-2.0\nunit-console-x86.exe" /labels /out=TestResult.txt /xml=TestResult.xml .\bin\Debug\TheWebApp.Specs.dll }
    exec { & "C:\path\to\SpecFlow\1.8.1\specflow.exe" nunitexecutionreport TheWebApp.Specs.csproj /out:SpecResult.html }
}

That last exec call to specflow.exe fails though, with:

The element <ParameterGroup> beneath element <UsingTask> is unrecognized. C:\Program Files (x86)\Jenkins\jobs\TheWebApp\workspace\Web\Sites\TheWebApp.nuget\nuget.targets

A bit of googling hints that maybe it's a problem with the msbuild version being used (e.g. here, here). But I have Framework "4.0" in my psake script, and the Specs project is targeting .NET Framework 4.0, and it builds fine in the build step, so I'm not sure why specflow seems to be using an earlier version of msbuild. Or is maybe the problem somewhere else?

Answer 1

This was the answer for me, from the SpecFlow Wiki:

Important for .NET 4.0 projects: Because specflow.exe is compiled for .NET 3.5, it cannot load .NET 4.0 assemblies by default. To generate this report for .NET 4.0 projects, you have to force specflow.exe to use the .NET 4.0 runtime by using the config file. Just copy the config below and create a specflow.exe.config file and put it next to your specflow.exe and you will be able to create the step definition report.

<?xml version="1.0" encoding="utf-8" ?> 
<configuration> 
    <startup> 
        <supportedRuntime version="v4.0.30319" /> 
    </startup> 
</configuration> 

Answer 2

I attempted to use the config file solution suggested above. It worked for testing locally, but as soon as I pushed my code to our CI environment it choked on it since the CI environment doesn't have that config file. We restrict out CI environment to only use clean versions of the various packages, so we didn't want to try to inject the special config into the CI server.

We noticed that SpecFlow works just fine with several of our .NET 4.0 projects without the special config file. After a little research, the actual 'problem' appears to be NuGet 2.1. Everything works fine for .NET 4.0 projects with NuGet 1.7.

Somewhere between 1.7 and 2.1 NuGet introduced new features in the NuGet.targets file that aren't supported by the older versions of MSBuild. Specifically the problem seems to be the <ParameterGroup> beneath element <UsingTask>, as explained by the error message.

A cursory glance at the targets file indicates that the section is responsible for keeping NuGet up to date. Removing this section completely resolves the issue in the same manner that adding the config file above does, albeit also removing the self-update functionality that is seems to provide. Given that the .targets file is committed to the repository, this solution also works on our CI environment with out any changes on the CI side.

It's not necessarily a better solution than ngm's, it's just a different one. Depending on your environment, this may be a preferable way to go, or perhaps not.