Unit testing code coverage - do you have 100% coverage?

Question

Do your unit tests constitute 100% code coverage? Yes or no, and why or why not.

Answer 1

There's a lot of good information here, I just wanted to add a few more benefits that I've found when aiming for 100% code coverage in the past

• It helps reduce code complexity

Since it is easier to remove a line than it is to write a test case, aiming for 100% coverage forces you to justify every line, every branch, every if statement, often leading you to discover a much simpler way to do things that requires fewer tests

• It helps develop good test granularity

You can achieve high test coverage by writing lots of small tests testing tiny bits of implementation as you go. This can be useful for tricky bits of logic but doing it for every piece of code no matter how trivial can be tedious, slow you down and become a real maintenance burden also making your code harder to refactor. On the other hand, it is very hard to achieve good test coverage with very high level end to end behavioural tests because typically the thing you are testing involves many components interacting in complicated ways and the permutations of possible cases become very large very quickly. Therefore if you are practical and also want to aim for 100% test coverage, you quickly learn to find a level of granularity for your tests where you can achieve a high level of coverage with a few good tests. You can achieve this by testing components at a level where they are simple enough that you can reasonably cover all the edge cases but also complicated enough that you can test meaningful behaviour. Such tests end up being simple, meaningful and useful for identifying and fixing bugs. I think this is a good skill and improves code quality and maintainability.

Answer 2

What I do when I get the chance is to insert statements on every branch of the code that can be grepped for and that record if they've been hit, so that I can do some sort of comparison to see which statements have not been hit. This is a bit of a chore, so I'm not always good about it.

I just built a small UI app to use in charity auctions, that uses MySQL as its DB. Since I really, really didn't want it to break in the middle of an auction, I tried something new.

Since it was in VC6 (C++ + MFC) I defined two macros:

#define TCOV ASSERT(FALSE)
#define _COV ASSERT(TRUE)

and then I sprinkled

TCOV;

throughout the code, on every separate path I could find, and in every routine. Then I ran the program under the debugger, and every time it hit a TCOV, it would halt. I would look at the code for any obvious problems, and then edit it to _COV, then continue. The code would recompile on the fly and move on to the next TCOV. In this way, I slowly, laboriously, eliminated enough TCOV statements so it would run "normally".

After a while, I grepped the code for TCOV, and that showed what code I had not tested. Then I went back and ran it again, making sure to test more branches I had not tried earlier. I kept doing this until there were no TCOV statements left in the code.

This took a few hours, but in the process I found and fixed several bugs. There is no way I could have had the discipline to make and follow a test plan that would have been that thorough. Not only did I know I had covered all branches, but it had made me look at every branch while it was running - a very good kind of code review.

So, whether or not you use a coverage tool, this is a good way to root out bugs that would otherwise lurk in the code until a more embarrasing time.

Answer 3

From Ted Neward blog.

By this point in time, most developers have at least heard of, if not considered adoption of, the Masochistic Testing meme. Fellow NFJS'ers Stuart Halloway and Justin Gehtland have founded a consultancy firm, Relevance, that sets a high bar as a corporate cultural standard: 100% test coverage of their code.

Neal Ford has reported that ThoughtWorks makes similar statements, though it's my understanding that clients sometimes put accidental obstacles in their way of achieving said goal. It's ambitious, but as the ancient American Indian proverb is said to state,

If you aim your arrow at the sun, it will fly higher and farther than if you aim it at the ground.

Answer 4

I usually manage to hit 93..100% with my coverage but I don't aim for 100% anymore. I used to do that and while it's doable, it's not worth the effort beyond a certain point because testing blindly obvious usually isn't needed. Good example of this could be the true evaluation branch of the following code snipped

public void method(boolean someBoolean) {
    if (someBoolean) {
        return;
    } else {
        /* do lots of stuff */ 
    }
}

However what's important to achieve is to as close to 100% coverage on functional parts of the class as possible since those are the dangerous waters of your application, the misty bog of creeping bugs and undefined behaviour and of course the money-making flea circus.

Answer 5

I generally write unit tests just as a regression-prevention method. When a bug is reported that I have to fix, I create a unit test to ensure that it doesn't re-surface in the future. I may create a few tests for sections of functionality I have to make sure stay intact (or for complex inter-part interactions), but I usually want for the bug fix to tell me one is necessary.