Continuous Integration for Haskell: Cabal TeamCity plugin!

By: on May 20, 2012

I’m happy to announce that my _Haskell Cabal TeamCity plugin_ is [available for
download][Cabal TeamCity plugin].

With this plugin you can practise continuous integration (CI) with your [Cabalised Haskell
projects][Cabal] using a CI server called [TeamCity][].

In case you haven’t heard of TeamCity, it’s a really neat piece of kit.
Internally we use TeamCity quite extensively to perform automated continuous
builds (and sometimes deployments) of our Maven, Ant, and NAnt-based projects.
It’s incredibly feature-rich, and has a very visual, clean and clear web interface.

[Cabal TeamCity plugin]:

Here’s a little sneak peek of it in action (with some tests I wrote for my little Minecraft tool):

I’m pretty pleased to make this great CI tool relevant to the Haskell community.

### Why this plugin?
CI servers such as [CruiseControl][], [BuildBot][], [Hudson][],
[Jenkins][], [TeamCity][] have out-of-the-box support for popular build systems like
Maven, Rake and NAnt allowing you to build projects written with their
associated mainstream programming languages like Java, Ruby and C# effortlessly.

Slightly more obscure build systems like [Cabal][] for Haskell were never going
to be supported out of the box, so decent CI options for Haskell have been few.

_Enter this plugin_. It provides basic level of Cabal support (packaging / build
system for Haskell projects) with the TeamCity CI server.

### What does it do?
The plugin saves Haskell developers from resorting to the catch-all command-line
(exec) mechanism to invoke their Cabal project builds. You get a whole bunch of extras too, including
informative build reporting, real-time status updates, basic
test reporting integration with Max Bolingbroke’s `test-framework`, and plenty of other TeamCity features that come for free.

Setting it up is quite effortless. Follow the Quickstart below or take a look at the [plugin webpage][Cabal TeamCity plugin] for more information.

### Quickstart
To use this plugin, you will need an installation of TeamCity.

- [Install the TeamCity Server and TeamCity Agent(s)][Install TeamCity] as on the TeamCity website
- [Follow installation instructions on the plugin website][Cabal TeamCity plugin]
- Satisfy the build Agent pre-requisites (see below)

##### Server pre-requisites
The TeamCity Server is only responsible for communicating with a number of build
Agents as well as presenting the user with a highly interactive web interface.

You will *not* need to install a Haskell compiler or libraries on this host.

##### Agent pre-requisites

The TeamCity agent will receive and process build jobs from the Server. Agents
must be capable of building the project. This includes having a Haskell compiler
(e.g. GHC) and `cabal-install` tool installed and accessible from `PATH`, as well
as all relevant package dependencies for the project to be built.

The easiest way to do the former is to install the [Haskell Platform][].

### How the plugin does its work
Any Haskell project packaged with Cabal can typically be built with the
following three commands.

`$ cabal configure`

`$ cabal build`

`$ cabal test`

The plugin essentially does this under the covers and a lot more.

It looks at the build log as the log lines are printed, notifying TeamCity of the status of
the build as the build runs through its ‘OpenAPI’.
It identifies and notifies TeamCity of what tests are being run so that TeamCity may provide
useful test statistics (such as success/failure rates) to the user.

The caveat is that test reporting is only supported for `test-framework`. Tests
run under another test framework or a custom test runner will simply not be recognised.

##### To Service Message or Not to Service Message
There is actually a better way of extracting information about the build than to
parse the build output log. TeamCity supports something called [service
messages][TeamCity service messages], which are essentially specially formatted
lines in the build output that notifies TeamCity of particular build events.

To accurately output these lines of output in the build is likely to require a number of intrusive changes to the bits involved in the build, such as `test-framework`. I’ve contacted Max Bolingbroke about this.

### Issues
A bulk of the work on this was done over a long weekend, so even though it’s worked quite well for me,
you might find it’s a bit rough around the edges.

You might find, for example that tests’ elapsed times are wrong. This is
unavoidable given the way that `test-framework` runs tests in parallel and the
build output is output all at the end.

A complete list of known issues due to this extrusive approach is documented on
the [plugin's GitHub home page][known issues].

##### Future work
Resolving the current issues will require a bit of change of tack. I would like
to develop an intrusive service message based implementation.

This approach can resolve many of the known issues. It could potentially report even parallel test runs’ elapsed times correctly using TeamCity’s so called ‘flowIds’.

Basically what I’m saying is, expect to discover some weird behaviour. *I need your help* to fix them!

### Alternative Haskell CI solutions
If you find that TeamCity’s free and commercial licenses are too restrictive, or
that this plugin doesn’t suit your needs, I include here a limited number of
alternative CI options for Haskell development.

- [Travis][] is an open source hosted CI server, probably more suitable for open source projects
- [scoutess][] is a CI effort recently announced by Alp Mestanogullari. This
seems to have a much deeper scope than this plugin, aiming to address dependency hell problems.
It’s been proposed as a [GSOC][] project
- use various command line runners…

A little fun fact, apparently the GHC project uses [GHC uses BuildBot][] for
continous integration and testing.

### Building other TeamCity plugins
A final word on TeamCity plugins. If you’re looking to build one,
instead of building the plugin from ground up with the [Maven TeamCity
Archetype][] (which lacked the agent side source structure) I found
[Unity3D TeamCity Runner][]. I used this as the basis of the plugin and
iteratively persuaded it into working with Cabal instead!

Good luck and have fun Haskelling!

[Industrial Haskell Group]:
[known issues]:
[Question answered]:
[Unity3D TeamCity Runner]:
[Maven TeamCity Archetype]: “Maven TeamCity Architype”
[Travis CI]: “Haskell support for Travis CI”
[CruiseControl]: “Cruise Control”
[Install Teamcity]:
[Haskell Platform]:
[TeamCity service messages]:
[GHC uses BuildBot]:


Post a comment

Your email address will not be published.


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>