Why is [AssemblyInitialize] and [AssemblyCleanup] being called twice in same test project assembly?

Question

I thought the whole purpose of these attributes was to run them only once per assembly. I have a simple class as follows:

[TestClass]
public class AssemblyIntegrationTestSetup
{
    public AssemblyIntegrationTestSetup() { }
    public TestContext TestContext { get; set; }

    [AssemblyInitialize]
    public static void SetupIntegrationTests(TestContext context)
    {
         WindowsServiceService.Instance.StartService("Distributed Transaction Coordinator");
    }

    [AssemblyCleanup]
    public static void TeardownIntegrationTests()
    {
          WindowsServiceService.Instance.StopService("Distributed Transaction Coordinator");
    }

}

However when I run the test suite the assembly-level Initialize and Cleanup methods execute twice. Here are the details about my environemnt:

1. All test classes are in the same project/assembly.
2. I have integration and unit tests separated by namespace.
3. For the integration tests, I am using MSTextExtensions to allow for rollback on database transactions.
4. I am also starting/stopping the MS SQL Server DTC service, which is required for the rollback ability. I wanted to do this once per test suite run (and the best compromise I found was to use the Assembly-level attributes). The code will work, but it executes twice.
5. If it matters, I'm using also the Microsoft Moles Framework in some of my tests.

The observed behavior is similar to:

AssemblyInitialize         

Class1.TestInitialize
Class1.TestMethod1
Class1.TestCleanup

AssemblyInitalize         <-- //This shouldn't be happening right?

Class2.TestInitialize
Class2.TestMethod1
Class2.TestCleanup

Class2.TestInitialize
Class2.TestMethod2
Class2.TestCleanup

Class5.TestInitialize
Class5.TestMethod1
Class5.TestCleanup

Class7.TestInitialize
Class7.TestMethod1
Class7.TestCleanup

//More random bouncing around then...

AssemblyCleanup 
AssemblyCleanup           <-- //This shouldn't be happening right?

Answer 1

From the MSDN Library article:

Important

This attribute should not be used on ASP.NET unit tests, that is, any test with [HostType("ASP.NET")] attribute. Because of the stateless nature of IIS and ASP.NET, a method decorated with this attribute might be called more than once per test run.

---

There are few knobs you can tweak in test runner. I would just punt the problem with a counter:

private int InitCount;

[AssemblyInitialize]
public static void SetupIntegrationTests(TestContext context)
{
     if (InitCount++ == 0) {
         WindowsServiceService.Instance.StartService("Distributed Transaction Coordinator");
     }
}

[AssemblyCleanup]
public static void TeardownIntegrationTests()
{
      if (--InitCount == 0) {
          WindowsServiceService.Instance.StopService("Distributed Transaction Coordinator");
      }
}

Answer 2

Well, this answer is a bazillion years late based on the original problem date but...

I discovered if the test agent (QTAgent32.exe) crashes or dies before the complete sequence finishes, then AssemblyInitialize (and maybe ClassInitialize and TestInitialize) would be called again. For example, put this in your [AssemblyCleanup] function and you'll see the behavior occur:

Process p = AutotestShared.RunProcess("cmd", "/c taskkill /t /f /im QTAgent32.exe", true);
p.WaitForExit();

So the moral of this story is: Check your cleanup functions to see if there are any crashes / corruption. Failures during cleanup don't show in the test report because the pass/fail assertions are already complete. But the problems it causes may show up in other ways.

Answer 3

In my case, it was a misbehavior of QTAgent32.exe process that was causing [AssemblyInitialize] to be executed twice. Restarting the host machine where my code was built and executed solved the issue for me.