Continuous Integration for Haskell: Cabal TeamCity plugin!

By: on May 20, 2012

I’m happy to announce that my Haskell Cabal TeamCity plugin is available for download.

With this plugin you can practise continuous integration (CI) with your Cabalised Haskell projects using a CI server called TeamCity.

In case you haven’t heard of TeamCity, it’s a really neat piece of kit. Internally we use TeamCity quite extensively to perform automated continuous builds (and sometimes deployments) of our Maven, Ant, and NAnt-based projects. It’s incredibly feature-rich, and has a very visual, clean and clear web interface.

Here’s a little sneak peek of it in action (with some tests I wrote for my little Minecraft tool):

I’m pretty pleased to make this great CI tool relevant to the Haskell community.

Why this plugin?

CI servers such as CruiseControl, BuildBot, Hudson, Jenkins, TeamCity have out-of-the-box support for popular build systems like Maven, Rake and NAnt allowing you to build projects written with their associated mainstream programming languages like Java, Ruby and C# effortlessly.

Slightly more obscure build systems like Cabal for Haskell were never going to be supported out of the box, so decent CI options for Haskell have been few.

Enter this plugin. It provides basic level of Cabal support (packaging / build system for Haskell projects) with the TeamCity CI server.

What does it do?

The plugin saves Haskell developers from resorting to the catch-all command-line (exec) mechanism to invoke their Cabal project builds. You get a whole bunch of extras too, including informative build reporting, real-time status updates, basic test reporting integration with Max Bolingbroke’s test-framework, and plenty of other TeamCity features that come for free.

Setting it up is quite effortless. Follow the Quickstart below or take a look at the plugin webpage for more information.

Quickstart

To use this plugin, you will need an installation of TeamCity.

Server pre-requisites

The TeamCity Server is only responsible for communicating with a number of build Agents as well as presenting the user with a highly interactive web interface.

You will not need to install a Haskell compiler or libraries on this host.

Agent pre-requisites

The TeamCity agent will receive and process build jobs from the Server. Agents must be capable of building the project. This includes having a Haskell compiler (e.g. GHC) and cabal-install tool installed and accessible from PATH, as well as all relevant package dependencies for the project to be built.

The easiest way to do the former is to install the Haskell Platform.

How the plugin does its work

Any Haskell project packaged with Cabal can typically be built with the following three commands.

$ cabal configure

$ cabal build

$ cabal test

The plugin essentially does this under the covers and a lot more.

It looks at the build log as the log lines are printed, notifying TeamCity of the status of the build as the build runs through its ‘OpenAPI’. It identifies and notifies TeamCity of what tests are being run so that TeamCity may provide useful test statistics (such as success/failure rates) to the user.

The caveat is that test reporting is only supported for test-framework. Tests run under another test framework or a custom test runner will simply not be recognised.

To Service Message or Not to Service Message

There is actually a better way of extracting information about the build than to parse the build output log. TeamCity supports something called service messages, which are essentially specially formatted lines in the build output that notifies TeamCity of particular build events.

To accurately output these lines of output in the build is likely to require a number of intrusive changes to the bits involved in the build, such as test-framework. I’ve contacted Max Bolingbroke about this.

Issues

A bulk of the work on this was done over a long weekend, so even though it’s worked quite well for me, you might find it’s a bit rough around the edges.

You might find, for example that tests’ elapsed times are wrong. This is unavoidable given the way that test-framework runs tests in parallel and the build output is output all at the end.

A complete list of known issues due to this extrusive approach is documented on the plugin’s GitHub home page.

Future work

Resolving the current issues will require a bit of change of tack. I would like to develop an intrusive service message based implementation.

This approach can resolve many of the known issues. It could potentially report even parallel test runs’ elapsed times correctly using TeamCity’s so called ‘flowIds’.

Basically what I’m saying is, expect to discover some weird behaviour. I need your help to fix them!

Alternative Haskell CI solutions

If you find that TeamCity’s free and commercial licenses are too restrictive, or that this plugin doesn’t suit your needs, I include here a limited number of alternative CI options for Haskell development.

  • Travis is an open source hosted CI server, probably more suitable for open source projects
  • scoutess is a CI effort recently announced by Alp Mestanogullari. This seems to have a much deeper scope than this plugin, aiming to address dependency hell problems. It’s been proposed as a GSOC project
  • use various command line runners…

A little fun fact, apparently the GHC project uses GHC uses BuildBot for continous integration and testing.

Building other TeamCity plugins

A final word on TeamCity plugins. If you’re looking to build one, instead of building the plugin from ground up with the Maven TeamCity Archetype (which lacked the agent side source structure) I found Unity3D TeamCity Runner. I used this as the basis of the plugin and iteratively persuaded it into working with Cabal instead!

Good luck and have fun Haskelling!

FacebookTwitterGoogle+

Post a comment

Your email address will not be published.

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>