How to measure code coverage with JaCoCo
If you’ve read the guide to writing your first Java unit test, then you already know the basics of unit testing and have even created your first unit test. You’re probably wondering how these tests can combine to create a comprehensive test suite (and how many of these tests you’ll need to write). In this chapter, we’ll talk about code coverage and how to measure it.
How many unit tests should I write?
This is a common question for new unit testers. There are many schools of thought about the answer, but the rule of thumb is “enough tests to ensure that serious regressions are not introduced as the code is modified.” So how do you measure this?
The most commonly used metric is code coverage. It’s not perfect, but it does tell you how much of the code you’ve written is covered by tests. It is widely known that high code coverage is not necessarily good; developers can write buggy code and then unit tests that pass with the buggy code. However, low code coverage is definitely bad, because it means your test suite is unlikely to catch issues that are introduced as the code is modified.
Based on this, it makes sense to measure the code coverage for test suites to help us identify code that is currently untested, as well as code that’s unused or potentially dead. There are a number of tools available for measuring code coverage, e.g. Cobertura and JaCoCo.
Getting Started
For the purposes of this tutorial, let’s use JaCoCo. Code coverage tools work by instrumenting the class files so they can record which lines are executed. With JaCoCo, there are two types of instrumentation:
- Default: This is where the class files are instrumented on-the-fly as they are called. The advantage of this is that it is the simplest to set up.
- Offline instrumentation: This is where all the class files are instrumented before running any of the tests. The advantage here is that coverage will be recorded in more circumstances. See the documentation for more details.
Given that offline instrumentation can give coverage information in more circumstances than the default, let’s set this up.
The first thing we need to do is to add the dependency to our pom.xml:
<dependency>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.3</version>
<scope>test</scope>
</dependency>Then we need to add the surefire plugin to the ‘build, plugins’ section of our pom.xml:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<systemPropertyVariables>
<jacoco-agent.destfile>$/jacoco.exec</jacoco-agent.destfile>
</systemPropertyVariables>
</configuration>
</plugin>Note that we have specified a location for the jacoco.exec file. This is the raw coverage file that is produced by Jacoco.
Next, we will add the JaCoCo plugin to the build, plugins section of our pom.xml:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.3</version>
<executions>
<execution>
<id>instrument</id>
<phase>process-test-classes</phase>
<goals>
<goal>instrument</goal>
</goals>
</execution>
<execution>
<id>restore-instrumented-classes</id>
<phase>test</phase>
<goals>
<goal>restore-instrumented-classes</goal>
</goals>
</execution>
<execution>
<!-- Ensures that the code coverage report for unit tests is created after unit tests have been run. -->
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>$/jacoco</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>So this is the bit that defines the process, essentially split into three parts:
- Instrument the classes being tested
- Restore instrumented classes
- Generate code coverage report
Step one happens before the unit tests are run. Steps two and three are run after the unit tests.
Now we are ready to see our first coverage report. We need to ensure that all the stages are run now that we have added extra steps from the plugin, so it’s best to run:
mvn clean testFrom here, we will get a report that looks like this:
We can find the HTML version of the report under target/site/jacoco (i.e. $/jacoco). You will notice that you can drill down into the package and the classes to see the coverage, down to individual lines:
There are three basic colors for the lines:
- Green: the line is completely covered by existing tests.
- Yellow: the line is partially covered. This means that it has been hit, but as in the example above, only one of the possible branches has been executed.
- Red: there is no coverage for this line.
There are more details on the coverage color coding and other info in the documentation.
The full source for this tutorial is available here.
Hopefully, this has been helpful in learning how to enable JaCoCo for your maven project.
