Unit test execution speed (how many tests per second?)

Question

What kind of execution rate do you aim for with your unit tests (# test per second)? How long is too long for an individual unit test?

I\'d be interested in knowing if

Answer 1

One of the most important rules about unit tests is they should run fast.

How long is too long for an individual unit test?

Developers should be able to run the whole suite of unit tests in seconds, and definitely not in minutes and minutes. Developers should be able to quickly run them after changing the code in anyway. If it takes too long, they won't bother running them and you lose one of the main benefits of the tests.

What kind of execution rate do you aim for with your unit tests (# test per second)?

You should aim for each test to run in an order of milliseconds, anything over 1 second is probably testing too much.

We currently have about 800 tests that run in under 30 seconds, about 27 tests per second. This includes the time to launch the mobile emulator needed to run them. Most of them each take 0-5ms (if I remember correctly).

We have one or two that take about 3 seconds, which are probably candidates for checking, but the important thing is the whole test suite doesn't take so long that it puts off developers running it, and doesn't significantly slow down our continuous integration build.

We also have a configurable timeout limit set to 5 seconds -- anything taking longer will fail.