What about code coverage

Last Updated on by

Post summary: Code coverage is measurement of what percentage of program source code is being executed by your tests. Code coverage is a nice to have, but in no case make code coverage ultimate goal!

Code coverage is useful metric to check what part of the code your tests are exercising. Is really depends on the tools to be used for gathering code coverage metrics but in general they work in similar fashion.

How it works

One approach is to instrument application under test. Instrumentation is to modify original executables by adding meta data so code coverage tool is able to understand which line of code is being executed. Other option is to run the application through code coverage tool. In both ways once application is running tests are being executed.

What tests to run

Tests that can be run to measure code coverage can be unit tests, functional automation tests or even manual tests. It doesn’t matter the important part is to see how our tests are impacting application under test.

Results

Once tests are finished code coverage metrics are obtained from code coverage tool. To have detailed information most of the tools take original source code and generate visual report with colour information which line is executed and which not. There are different levels of coverage, method, line, code block, etc. I personally prefer lines and when speaking further I’ll mean lines of code executed.

Benefits

Code coverage information is equally useful for QA and developers. QA analyse code that has not being executed during tests to identify what tests conditions they are missing and improve their tests. Developers analyse the results to identify and remove dead or unused code.

When to do it

Code coverage is a nice to have activity. We can live with or without it. It is useful for big matured products where there is automation test suites. You can also do them against your tests code in order to optimise it. Removing dead code optimises the product and makes its maintenance easier. I would say doing it too often should be avoided. Everything depends on context but for me best is once or twice a year.

What does code coverage percentage means

In one word – nothing. You may have 30% of code coverage but to cover most important user functionality and bug rate to be low and you may have 90% with dummy tests made especially to exercise some code without the idea of actually testing it. I was lucky to work on a project where developers keep the code tidy and clean and my tests easily accomplished 81% just by verifying all user requirements. I may say 80-85% is the maximum you may get.

Pitfalls

Do not ever put code coverage as important measurement or key performance indicator (KPI) in your testing strategy. Doing so and making people try to increase the code coverage will in most cases result in dummy tests made especially this percentage to go up. Code coverage is in most cases insignificant aspect of your testing strategy.

How to do it

Tutorial how to do code coverage for Java with JaCoCo can be found in Code coverage of manual or automated tests with JaCoCo post. Tutorial how to do code coverage for Java with OpenCover can be found in Code coverage of manual or automated tests with OpenCover for .NET applications post.

Conclusion

Code coverage is interesting aspect of testing. It might enhance your tests if done wisely or it can ruin them. Remember tests scenarios should be extracted from user requirements and features. Code coverage data should be used to see if you have some blind spots reading the requirements. It can help developers remove dead or unused code.

If you find this post useful, please share it to reach more people. Sharing is caring!
Share on FacebookShare on LinkedInTweet about this on TwitterShare on Google+Email this to someone
Category: C#, Java, Non-functional | Tags: