April, 2012: Hylomorphisms through lazy streams

A fold is a higher-order function familiar to many programmers. It’s a function that takes some structure as input, and some function that converts part of that structure into a value. Evaluating the fold then recursively applies the given function to the structure, resulting in some return value. Variants exist which supply an initial “seed” value, or just use the first element of the structure as seed. For instance, given a list of numbers #(1 2 3 4 5) , and a folding function #+ , #(1 2 3 4 5) inject: 0 into: #+ will calculate the sum of the numbers.

Perhaps less well known is the unfold, at which we are going to look today.

(more…)

April, 2012: Open-source attribution & contribution

There are two things that I keep seeing when I come across open source projects, admittedly mostly on GitHub.

1. Code which is intended to be open-source but which doesn’t have any licence on it.
2. Attribution and copyright around contributions is lacking.

These two issues have bothered me with enough frequency lately that maybe it’s worth explaining in long form. One of them might be seen to be amateur copyright lawyering, at least, although I haven’t been able to find a definitive answer on the issue of contributions. The attribution part is important, in either case, and it’s linked to the copyright.

(more…)

April, 2012: Persuading mercurial-server to work on Windows

A few years back, needing decent authentication and key management for Mercurial repositories, we wrote mercurial-server. Recently, we were in the position of wanting to use this with one of our clients, but this time it would need to run under Windows. This was a bit of a problem as it was designed for a Unix-type environment. On the other hand, it’s mostly written in Python, and we could use Cygwin, so how bad could it be?

As it turns out, not too awful. I needed to make a few small modifications, and we’ve now managed to deploy a working mercurial-server under Windows. (more…)

April, 2012: The Fallacies of Distributed Computing

I recently helped a client take ownership of an application stack produced by a third party. There were various components to this, but the most interesting aspect allowed users to interact with each other by manipulating images in a browser. This was achieved using a small and simple NodeJS application to send events between the browsers.

We were aware of a few severe problems shortly before our involvement, and during the handover meeting I asked the developer how he had fixed them. He replied that if two or more users tried to manipulate the same object at the same time, it would cause the browser code to crash as multiple conflicting change events would be sent. He therefore added a new event type that was sent out when an action started, e.g., on mouse down. This indicated that all the other browsers should lock their copies of the object in question until the action was finished.

I gently suggested that this might not work well over a network, to which he responded that in testing the lock events took less than 10ms to arrive, so the odds of two people trying to do the same thing within that period were so small as to not be worth worrying about.

Again, I very gently suggested that the benchmark was unlikely to hold across the Internet. He told me that he had tested it on several networks including over the public Internet with someone in the US and it was still good. I gave up at this point, and just noted to the client that this was something that would need attention fairly promptly.

This assertion that there was a latency of 10ms between the UK and the US is clearly false. Even if you could set-up a perfect vacuum between London and New York, it would still take 19ms for light to travel that distance. However, the numbers are irrelevant – there is latency and it must be considered when trying to build a reliable distributed system regardless of magnitude. This and seven other issues were famously covered in the “Eight Fallacies of Distributed Computing”, assembled by Peter Deutsch, James Gosling and others.

Anyone building a distributed application should have these fallacies in mind, and this paper has a good explanation of them with a very readable commentary. NodeJS made this shared browser experience extremely effective and enticingly simple to construct, but even NodeJS apps are not isolated from the effects of the infrastructure they utilise. In fact, until TCP over entangled photon pairs hits the shelves, these fallacies will be relevant regardless of your framework.