technology from back to front

Archive for June, 2007

Ambient authority and Sleepycat Java Edition and stateful services

I’ve recently written an RMI service which has state – transactions. The service is implemented using Sleepycat Java Edition collections, and the transactions map to sleepycat transactions.

The StoredMap class depends on ambient authority: it determines the current transaction from the thread. The methods will all be invoked in on separate threads, so we need to deal with this somehow. This should be pretty simple – surely there will be a ‘join’ method – to join the current thread to the transaction. Nope. So now what?

I’ll just have to keep a thread running for the transaction, and execute transactional code in that thread.

In Java 5, we get Executors and Futures in java.util.concurrent, which make this very simple to implement. In fact, we can add some sugar – I can generate a proxy class for an interface, which executes its methods inside a single thread.

Anyway, I have, and added it to the LShift Java Library. It even mangles stack traces into something you can read.


Updated AJAX Erlang Jukebox

Erlang Jukebox ScreenshotOur jukebox (mentioned previously) received an update yesterday.

To download the code,

There is a little bit of documentation available, and you can browse the code.


Deadlocks are annoying

Java’s concurrency model provides a sophisticated menu of ways in which to shoot yourself in the foot. Many styles and many variations are available. To give a taste of some of the delicacies on offer, here’s the essence of a problem I found in some code I’d written recently.

Let thread one perform the tasks:

  1. lock C
  2. wait for event 1
  3. unlock C
  4. lock C
  5. wait for event 2
  6. unlock C

Let thread two perform the tasks:

  1. signal event 1
  2. lock C
  3. unlock C
  4. signal event 2

When we were trying to pin down the issue, we found that the code would run through to the end about 10% of the time. The other runs locked up with thread one blocked at step 5, and thread two at step 2.

Java doesn’t make any promises about fairness: specifically, thread one’s release of the lock before reacquiring the same lock need not cause a thread switch, even when the runtime knows there’s another thread waiting for that lock.

There are solutions: both STM and shared-nothing message passing are models much easier to reason about. Let’s hope future revisions to the Java system raise the level of discourse.


The article about AMQP we’ve all been waiting for

AMQP is an open messaging protocol intended to support the high-volume, high-reliability applications
used in the financial and telecoms industries (among others). Some might see it as the next natural layer to
go on top of TCP/IP – that’s as close as I can get to describing it in a single phrase, anyway.

This article about it is quite illuminating:

The author is John O’Hara, principal instigator of AMQP and distinguished engineer at JP Morgan, who
actually use AMQP-based software for production applications. (Given the ludicrously demanding environment of
investment banking, this use-case tells its own story regarding the maturity and reliability of the protocol.)

The article gives a very readable description of AMQP, the design ideas that went into it, and the
current state of play as regards available implementations and likely adoption/usage scenarios. To pick a few stand-out points:

(*) Considerable thought has gone into designing the protocol for maximum flexibility, interoperability with existing systems,
and ease of adoption. As Mr O’Hara comments, the only “political constraint” (i.e. expedient holdover) that went into the design was compatibility with the already popular JMS messaging standard.

(*) A central idea of AMQP is to decouple the transport and queueing model. The article describes an extended analogy
with a nationwide postal service. Outgoing messages, adorned with headers, correspond to labelled envelopes. Exchanges
correspond to postboxes acting as a front end to configurable sorting/delivery systems. Queues correspond to holding areas
where mail is stored for the addressee to collect. All the machinery in between is user-definable and can be dynamically reconfigured.

(*) There are already several “industry strength” open-source broker/client implementations available. The article describes
some plausible, low-risk ways for organizations to experiment with AMQP using these.

(*) A particularly promising development is that the widely used trading protocol FIX may well use AMQP as an underlying transport in its next version. This is an attractive route for the FIX community because of perceived deficiencies in the currently implemented session layer. At a stroke, this would validate AMQP as a next-generation messaging standard and accelerate its
adoption by the financial services industry.


Why does everything on the web require registration?

Some sites or services, quite reasonably, need to know who I am (and that I really am that person, to some acceptable level of verifiability). It’s usually because they hold data on my behalf, and neither me nor they want anyone else getting at that data.

But why does InfoQ require me to register to download a free PDF?
They say,
… we’re happy to offer a free version for download, to get this knowledge in as many peoples hands as possible;
however, I had to complete a long form (to which I mostly invented answers), then verify my email address, then navigate back to the page. I hardly see how that is compatible with the stated aim.

After running into that, I almost despaired when I went to download the official MySQL JDBC driver.
Almost: there’s actually a link below the register (or log in) options saying “No thanks, just take me to the downloads!”.

Yay for!


Folds and continuation-passing-style

In normal, direct-style programming in (mostly-)functional languages such as scheme and ML, folding is an operation that crops up all the time in various guises. Most list-manipulation libraries for such languages include implementations of left-fold and right-fold as standard. But what about the situation when you’re programming in continuation-passing style (CPS), such as when you’re writing (or trying to write) a metacircular evaluator? Library support for continuation-passing folds isn’t nearly as common.

Here’s the direct-style left-fold function:

(define (foldl kons knil xs)
  (if (null? xs)
      (foldl kons (kons (car xs) knil) (cdr xs))))

and here’s the continuation-passing left-fold function:

(define (foldl-k kons knil xs k)
  (if (null? xs)
      (k knil)
      (kons (car xs) knil (lambda (v) (foldl-k kons v (cdr xs) k)))))

Note that kons takes three arguments here, where in the direct-style version, it takes two.

One benefit of having CPS folds available is that they expose more control over the loop. For instance, using a normal fold, there’s no way to terminate the iteration early, but using a CPS fold, your three-argument kons routine can simply omit invoking its continuation parameter (presumably choosing some other continuation to run instead). This means that operations like (short-circuiting) contains?, any, and every can be written with CPS fold, but not with plain direct-style fold:

(define (contains? predicate val elements)
  (foldl-k (lambda (elt acc k)
             (if (predicate elt val)
                 #t ;; note: skips the offered continuation!
                 (k acc)))
           (lambda (v) v)))

New Timetracker release

A long-awaited new timetracker release is now available in LShift’s Open Source section.

The main improvements are:

– The user interface is richer – this includes proper columns reordering, helpful colouring for fields and some integration with bugzilla.

– There is better HSQLDB schema consistency check and customization is much easier. We added more documentation on how to personalise your own timetracker.

We also resolved several bugs, so this version is much more stable.

You are more than than welcome to write about any of your experiences of using the new timetracker — just leave a comment or email us at timetracker(AT)lshift(DOT)net.




You are currently browsing the LShift Ltd. blog archives for June, 2007.



2000-14 LShift Ltd, 1st Floor, Hoxton Point, 6 Rufus Street, London, N1 6PE, UK+44 (0)20 7729 7060   Contact us