Use Clover to generate code coverage reports of your Integration/Automation Tests

clover is a great tool for generating code coverage reports from your unit tests. it can be executed as a plugin in eclipse, maven or ant.
however, not everyone knows that it can also be used to collect coverage data of integration tests.

• this post explains how to collect coverage data with clover at runtime.
• this post assumes that you already know what are unit and integration tests.
• this post assumes that you know what clover is, and already used it either with eclipse, ant or maven.
  

* let me assured you that even though the directions bellow seems complicated and clumsy at first, after doing them once or twice it is really easy to repeat them.

motivation
the default action of clover is to gather code coverage information during build time or compile time. therefore, this information includes just the coverage data created by unit tests.

if you are developing web applications, you probably use more technologies to test your applications beside unit tests. these technologies may include httpunit/ htmlunit or automation technologies (like selenium). these technologies do not work at build time, they can only work during run time, where a web server is up and running and http calls are made.
as a result, the code coverage made during build time is not reflecting the actual code coverage. we should be able to test the coverage while a server is running.

the idea
the idea is that we will first run clover regularly during build time. we will than take the clover artifacts, put them in our server and then run the integration tests.
while running the integration tests, the clover database will be updated and we would be able to generate reports from it which will reflect both unit and integration tests.

step 1 – preparation

• make sure that you have a web/application server (tomcat/jboss/weblogic…) with your web application already deployed.
• execute clover on your application as you would normally do (either by compiling the code on eclipse or by building with maven or ant, it doesn’t matter).
  
  
  the result of this action would be:
  
  
  1. clover db files.
     one of the outcome of executing the clover on your code are the db files.
     the db files hold all the information about your code and the coverage itself.

     the location of those files may change depending on the way you use clover and according to the way you configured clover in your environment.

     this is how the files looks like
     

     

     the .db file holds the information regarding your code (classes, methods and so on). all the other files hold all the coverage data.

     it is important that you will locate those files because we are going to use them in the next step.

  2. an instrumented code.
     another outcome of clover is that it instruments your code. a clover call is injected into each method so it would be reported in the coverage calculation.
     we will need this code. we will use this instrumented code in runtime to update the coverage data.

     if you use eclipse than the generated classes would be instrumented. if you use maven or ant than most chances are that a jar with all the instrumented code would be generated separately.
     search the instrument code jar. again, i can’t tell you exactly where it is located, but usually it generates a jar with a ‘clover’ postfix. example: if your jar name is my_app.jar, than the generated instrumented code jar will probably be something like my_app-clover.jar.
     so you will need to do some detective work here to find the instrumented classes/jar. if you are not sure whether the classes are instrumented or not, just decompile one of them with jad and search for the word clover inside of it.

  3. a code coverage report.
     this is a report with the unit test coverage. we don’t really need this, but it would be good so that we would be able to compare it with the report we will generate at the end.

step 2 – updating the server
the next steps are very important, please make sure you do them properly.

• replacing the existing application jar/classes with the instrumented jar/classes.
  take the instrumented jar/classes that were created by the clover and add it to your server’s classpath instead of the original jar/classes.
  the instrumented code will cause the clover db to be updated with the runtime data.
• adding clover jars and license.
  since we will use clover on runtime we will need also the clover jars in our server’s classpath.
  so add the clover own jars to your server’s classpath. if you are using clover with eclipse than these jars are located in the plugin folder of eclipse. if you are using maven than they will be loacted in your repository.

  * make sure you are using the same version of clover in your server as you used to generate the db files and instrumented code, otherwise it will not work and you will get error messages.
  also add the clover license file to the same location as the jars.

• adding the clover java argument.
  add the following java arg -dclover.initstring.basedir={ location of the db files that were created by the clover } .
  
  * notice – the path you have entered above is the path of the folder which contains the db files.
  
  example:
  -dclover.initstring.basedir=c:/workspace/mywebapp/target/clover.
  this java argument is used by the clover to locate the db files and update them.

step 3 – restarting the server and running the tests
now that hopefully all is set properly all that you need to do now is to restart your server and than running your integration tests.
the tests should trigger the instrumented code which will call the clover api’s and will update the clover db.

while running the tests:
1. look at your log/console and search for error messages from clover.
2. look at the folder which holds the clover db files. if everything is going as it should, new files will be created in this folder while running the tests.
   

if not everything is going well the first time, don’t discourage, just go over each of the steps again.

step 4 – generating an updated report
if everything went well and new files were created in the db folder than that means you just need to generate a new report.
if you are using clover with eclipse than you can simply push the reload button to reload the coverage data.
if you are using maven or ant you can execute just the task which generates the report.
another way is to use the clover htmlreporter to generate a report easily.

now compare the new report to the old report. you should see that the new report coverage is much bigger than the old one since it contains also the integration tests coverage.

* notice that not all the data is updated, even though the percentages are being updated, for some reason the calls counter does not.

to summarize. as mentioned; yes, these instructions seems a bit complicated but after you succeed the first time, it is very easy to repeat it. in the company i work for we even made this whole process automatic and we are able to generate a full coverage report with unit and integrated tests combined.

source: http://www.aviyehuda.com/2011/12/use-clover-to-generate-code-coverage-reports-of-your-integrationautomation-tests/