I like to write unit tests for my code and I also like to know whether my unit tests are actually testing my code. In Java I would use Maven and Cobertura to measure how
much of my code is actually exercised, and when I bend my mind around Haskell I use HPC.
I have been experimenting with Clojure and it has a nice testing library but the only way I have found to measure coverage is to use Cobertura and squint at the generated class names to work out what I haven’t covered. So I have started on the road to creating a coverage tool for Clojure written in Clojure. This blog entry contains my initial steps and design sketches towards creating the tool.
Ideally I’d like to measure my coverage like this:
That isn’t functional code yet and may not be what I finally produce, it might not even be valid Clojure since I haven’t even bashed it into a REPL yet.
As Clojure is a Lisp, and we are testing our own code – i.e. we have the source code – we should be able to measure the coverage using Clojure. This can achieved by rebinding the functions under test with instrumented functions;
this is what I envisage the cover function is doing:
Which can be achieved using a binding function:
So assuming I can write the cover function I will need to instrument a function. Let’s start simply and ignore the complicated stuff such as conditional statements and multiple bodies until later. If we have this simple function
I would like an instrumented version to wrap each s-expression with a function that records that the s-expression has been called, for now we will just print a message.
Testing that in the REPL seems to reveal that I am moving in the right direction:
To convert the first function to the instrumented function I need a couple of auxiliary functions and a macro:
If you know some Clojure that code demonstrates some of the problems I haven’t solved yet when measuring coverage, for example the :else clause will need to be covered as well as functions in binding statements. Testing that code in the REPL gives us a reasonable result:
So if I now replace the record function with something else I may be able to generate some coverage for a function. I am going to use a map held by a ref to store the coverage information, a function to register a particular
s-expression with the ref and a function to increment the count.
Lets test that:
If I modify our wrapping functions I can create the coverage records ready to be incremented.
Testing that code:
And that is as far as I have got. I need to change my record function to increment the coverage records, make my wrap function cope with conditionals and other complexities and write my rebinding function. Hopefully next month I will have moved it along a little bit further.