June, 2010: Another design sketch for Clojure coverage tool

Previously (here) I sketched out a design that went some way to being a coverage tool for Clojure code. The result was a macro that when used to define a function, resulted in an instrumented function that tallied every time the code was called. This time I am going to sketch the beginnings of a design that doesn’t use a macro.

To start with here is a slight re-write of the structure I used last time to record the coverage:

(def coverage-records (ref nil))

(defn add-record [fn-name record]
  (dosync (alter coverage-records assoc (str fn-name) record)))

(defn get-record [fn-name]
  (get @coverage-records (str fn-name)))

(defn covering [form fn-name]
  (dosync (alter (get-record fn-name) assoc form 0)))

(defn- inc-map [map key]
  (if (contains? map key)
    (assoc (dissoc map key) key (inc (get map key)))
    map))

(defn inc-coverage [form fn-name]
  (dosync (alter (get-record fn-name) inc-map form)))

This code uses a single map to store a coverage record, another map, for each function that is being measured. Functions are provided for adding records and incrementing the coverage.

I am using the same set of functions to wrap and instrument the s-expressions that define the function, with a few minor modifications. I am indexing the s-expressions with a number so as to distinguish between any identical s-expressions within the function. This set of functions is still incomplete since there are lots of Clojure code that will get wrapped incorrectly, I will be rectifying this sometime in the near future.

(declare wrap-seq)
(defn wrap
  [form idx fn-name]
  (cond
    (seq? form) (
      let [key (str idx ":" form)]
        (covering key fn-name)
        (list ‘do `(inc-coverage ~key ~(str fn-name)) (wrap-seq form idx fn-name)))
    :else form))

(defn- indexed [coll] (map vector (iterate inc 0) coll))

(defn wrap-seq [coll count fn-name]
  (for [[idx elt] (indexed coll)] (wrap elt (+ count idx) fn-name)))

The previous article used a macro to generate Clojure code that was instrumented, this time I will load the source code for the function, wrap it and then evaluate it. This is done by the get-source function in clojure.contrib.repl-utils combined with a call to format and load-string, My code that interprets the source code is fragile and needs more work, it won’t handle functions with multiple bodies for example, but it demonstrates the principle. I am using a structure to hold the instrumented function and the coverage record for this function.

(defstruct wrapper :wrapped-fn :coverage-record)

(defn wrap-fn
  [f cv-rec]
  (let [s (read-string (get-source f))
        fn-name (first (drop 1 s))
        args (first (drop 2 s))
        body (last s)]
    (add-record fn-name cv-rec)
    (load-string (format “(fn %s %s)” args (wrap body 0 fn-name)))))

(defn wrap-function [f]
  (let [coverage-record (ref nil)]
    (struct wrapper (wrap-fn f coverage-record) coverage-record)))

So now we can try the code out at the REPL:

user=> (def x (wrap-function 'test1))   
#'user/x
user=> x
{:wrapped-fn #, :coverage-record #}

Here I have wrapped the function test1 and we can see the coverage structure returned, consisting of the wrapped function and the coverage record, keyed by the single s-expression with a count of 0. If I now use the wrapped function like this and examine the coverage structure the count should increment to 1.

user=> ((:wrapped-fn x) 1 2)            
-1
user=> x
{:wrapped-fn #, :coverage-record #}

So we now have a partially functional coverage tool and no macros have needed to be written. To complete this tool I just need to tidy up the wrapping and source code reading functions and provide some sort of binding macro so that we can call the fn with

(test1 1 2)

instead of

((:wrapped-fn x) 1 2)

. Hopefully, in my next blog entry I will have completed it!