Unit testing code coverage - do you have 100% coverage?

Question

Do your unit tests constitute 100% code coverage? Yes or no, and why or why not.

Answer 1

Yes we do.

It depends on what language and framework you're using as to how easy that is to achieve though.

We're using Ruby on Rails for my current project. Ruby is very "mockable" in that you can stub/mock out large chunks of your code without having to build in overly complicated class composition and construction designs that you would have to do in other languages.

That said, we only have 100% line coverage (basically what rcov gives you). You still have to think about testing all the required branches.

This is only really possible if you include it from the start as part of your continuous integration build, and break the build if coverage drops below 100% - prompting developers to immediately fix it. Of course you could choose some other number as a target, but if you're starting fresh, there isn't much difference for the effort to get from 90% to 100%

We've also got a bunch of other metrics that break the build if they cross a given threshold as well (cyclomatic complexity, duplication for example) these all go together and help reinforce each other.

Again, you really have to have this stuff in place from the start to keep working at a strict level - either that or set some target you can hit, and gradually ratchet it up till you get to a level you're happy with.

Does doing this add value? I was skeptical at first, but I can honestly say that yes it does. Not primarily because you have thoroughly tested code (although that is definitely a benefit), but more in terms of writing simple code that is easy to test and reason about. If you know you have to have 100% test coverage, you stop writing overly complex if/else/while/try/catch monstrosities and Keep It Simple Stupid.

Answer 2

No for several reasons :

• It is really expensive to reach the 100% coverage, compared to the 90% or 95% for a benefit that is not obvious.
• Even with 100% of coverage, your code is not perfect. Take a look at this method (in fact it depends on which type of coverage you are talking about - branch coverage, line coverage...):

public static String foo(boolean someCondition) {
    String bar = null;
    if (someCondition) {
        bar = "blabla";
    }
    return bar.trim();
}

and the unit test:

assertEquals("blabla", foo(true));

The test will succeed, and your code coverage is 100%. However, if you add another test:

assertEquals("blabla", foo(false));

then you will get a NullPointerException. And as you were at 100% with the first test, you would have not necessarily write the second one!

Generally, I consider that the critical code must be covered at almost 100%, while the other code can be covered at 85-90%

Answer 3

No, because there is a practical trade-off between perfect unit tests and actually finishing a project :)

Answer 4

It is seldom practical to get 100% code coverage in a non-trivial system. Most developers who write unit tests shoot for the mid to high 90's.

An automated testing tool like Pex can help increase code coverage. It works by searching for hard-to-find edge cases.

Answer 5

Yes, I have had projects that have had 100% line coverage. See my answer to a similar question.

You can get 100% line coverage, but as others have pointed out here on SO and elsewhere on the internet its maybe only a minimum. When you consider path and branch coverage, there's a lot more work to do.

The other way of looking at it is to try to make your code so simple that its easy to get 100% line coverage.

Answer 6

On a new project I practice TDD and maintain 100% line coverage. It mostly occurs naturally through TDD. Coverage gaps are usually worth the attention and are easily filled. If the coverage tool I'm using provided branch coverage or something else I'd pay attention to that, although I've never seen branch coverage tell me anything, probably because TDD got there first.

My strongest argument for maintaining 100% coverage (if you care about coverage at all) is that it's much easier to maintain 100% coverage than to manage less than 100% coverage. If you have 100% coverage and it drops, you immediately know why and can easily fix it, because the drop is in code you've just been working on. But if you settle for 95% or whatever, it's easy to miss coverage regressions and you're forever re-reviewing known gaps. It's the exact reason why current best practice requires one's test suite to pass completely. Anything less is harder, not easier, to manage.

My attitude is definitely bolstered by having worked in Ruby for some time, where there are excellent test frameworks and test doubles are easy. 100% coverage is also easy in Python. I might have to lower my standards in an environment with less amenable tools.

I would love to have the same standards on legacy projects, but I've never found it practical to bring a large application with mediocre coverage up to 100% coverage; I've had to settle for 95-99%. It's always been just too much work to go back and cover all the old code. This does not contradict my argument that it's easy to keep a codebase at 100%; it's much easier when you maintain that standard from the beginning.