When I first started writing code with a TDD approach several years ago, I worked under the false impression that having high code coverage was important. I thought that in order to have a reliable and maintainable system I needed to have tests that touched every path through my codebase.

I used to run a coverage tool after every build to see whether the coverage percentage had gone up. If not, then it’s time to add more tests. If you have 100% code coverage and all your tests are passing, then this must surely mean that the application is doing everything correctly, right? Wrong.

The problem with code coverage as a metric for quality is that it’s misleading. Just because a method is covered by unit tests, it doesn’t mean that these tests are correct. As an example, imagine an application has a requirement to calculate the product of two integers, so a developer writes the following code:

public int CalculateProduct(int num1, int num2) {
  return num1 + num2;
}
 
[Test]
public void CalculatesProduct() {
  var result = CalculateProduct(10, 20);
  Assert.AreEqual(30, result);
}

This method has 100% coverage. The problem is that it’s completely wrong.

The developer clearly hasn’t had enough coffee as he’s misunderstood the requirement – the numbers should be multiplied, not added. 

So the developer fixes his code, but what about boundary conditions?

If we were to pass int.MaxValue as one of the arguments to this method, then the result is going to overflow and give back what is probably an undesirable result, yet this could be a perfectly possible scenario which may require testing for (depending on requirements).

If we were to rely solely on code coverage metrics to see whether our code is well tested then we may not bother to write any other tests as our single test covers 100% of the code paths through the method.

Now, this isn’t to say that code coverage metrics don’t have value – they do. Code coverage is useful because it shows you which parts of your codebase aren’t tested. Sometimes this might be important (maybe you’ve forgotten to write tests for a critical piece of functionality), but other times it might not be an issue.

Either way, it’s useful to be able to see which pieces of code are untested, but this doesn’t mean that the code that does have tests is correct, nor does it mean that the untested code needs to have tests (but that’s the subject for another post).

In summary, while code coverage is a useful metric it is also misleading. Just because code is well covered, it doesn’t mean that the code or the tests are correct.