How do you unit test a unit test? [closed]

Answer 1

Unit testing works very similar to double entry book keeping. You state the same thing (business rule) in two quite different ways (as programmed rules in your production code, and as simple, representative examples in your tests). It's very unlikely that you make the same mistake in both, so if they both agree with each other, it's rather unlikely that you got it wrong.

How is testing going to be worth the effort? In my experience in at least four ways, at least when doing test driven development:

• it helps you come up with a well decoupled design. You can only unit test code that is well decoupled;
• it helps you determine when you are done. Having to specify the needed behavior in tests helps to not build functionality that you don't actually need, and determine when the functionality is complete;
• it gives you a safety net for refactorings, which makes the code much more amenable to changes; and
• it saves you a lot of debugging time, which is horribly costly (I've heard estimates that traditionally, developers spend up to 80% of their time debugging).

Answer 2

How does one test a test? Mutation testing is a valuable technique that I have personally used to surprisingly good effect. Read the linked article for more details, and links to even more academic references, but in general it "tests your tests" by modifying your source code (changing "x += 1" to "x -= 1" for example) and then rerunning your tests, ensuring that at least one test fails. Any mutations that don't cause test failures are flagged for later investigation.

You'd be surprised at how you can have 100% line and branch coverage with a set of tests that look comprehensive, and yet you can fundamentally change or even comment out a line in your source without any of the tests complaining. Often this comes down to not testing with the right inputs to cover all boundary cases, sometimes it's more subtle, but in all cases I was impressed with how much came out of it.

Answer 3

Most unit tests, test assumptions. In this case, the discount price should be the price minus the discount. If your assumptions are wrong I bet your code is also wrong. And if you make a silly mistake, the test will fail and you will correct it.

If the rules change, the test will fail and that is a good thing. So you have to change the test too in this case.

As a general rule, if a test fails right away (and you don't use test first design), either the test or the code is wrong (or both if you are having a bad day). You use common sense (and possilby the specs) to correct the offending code and rerun the test.

Like Jason said, testing is security. And yes, sometimes they introduce extra work because of faulty tests. But most of the time they are huge time savers. (And you have the perfect opportunity to punish the guy who breaks the test (we are talking rubber chicken)).

Answer 4

I see unit tests and production code as having a symbiotic relationship. Simply put: one tests the other. And both test the developer.

Answer 5

Remember that the cost of fixing defects increases (exponentially) as the defects live through the development cycle. Yes, the testing team might catch the defect, but it will (usually) take more work to isolate and fix the defect from that point than if a unit test had failed, and it will be easier to introduce other defects while fixing it if you don't have unit tests to run.

That's usually easier to see with something more than a trivial example ... and with trivial examples, well, if you somehow mess up the unit test, the person reviewing it will catch the error in the test or the error in the code, or both. (They are being reviewed, right?) As tvanfosson points out, unit testing is just one part of an SQA plan.

In a sense, unit tests are insurance. They're no guarantee that you'll catch every defect, and it may seem at times like you're spending a lot of resources on them, but when they do catch defects that you can fix, you'll be spending a lot less than if you'd had no tests at all and had to fix all defects downstream.

Answer 6

Even if you do not test your code, it will surely be tested in production by your users. Users are very creative in trying to crash your soft and finding even non-critical errors.

Fixing bugs in production is much more costly than resolving issues in development phase. As a side-effect, you will lose income because of an exodus of customers. You can count on 11 lost or not gained customers for 1 angry customer.