technology from back to front

Archive for September, 2007

kill daemon from Planet -9

Someone has put programmers’ fridge magnet poetry on our fridge. I suspect it was a well-targetted marketing freebie, like the beanbag penguin that sits looking out onto Old Street. In any case, it has already attracted the attention of several clearly very talented individuals:

gorgeous dangling cyberspace
      whisper foo

and this ironic gem

   greasy hacker @ deprecated network
         tiny bucket

             spawn bandwidth


Fridge poetry

My favourite, though, is the person that has arranged some magnets into a pair of parens, with the single atom TRUE enclosed.

Up close, it is a subtle play on self-description, a visual pun on syntactic abstraction; but macroscopically, the seemingly trivially-evaluated expression brings recursion into the fold – its position and spelling suggests that there is more yet to be unravelled, and poses the question “How do we define truth?”.


Shifty needs a Programme / Business Manager

Shifty (LShift’s Guru and Avatar) is looking for an experienced Programme Manager / Business Manager with the nous and ambition to drive a truly innovative company. We’ve found that a traditional approach to running projects is as much of a hindrance as a help, so we need someone to report directly to the board, capable of defining, implementing and delivering new ways of working in a thoroughly agile environment. Do you fancy the challenge of ensuring that our systems are as smart as our technical offering?

Salary: £38 – £45k plus discretionary bonus, based in Central London.

You’ll work across five key areas:

  • Client Engagement: Put simply, this is about helping clients get the best out of their relationship with LShift. Strategically, this involves refining and communicating our proposition as a business; operationally, it means developing and improving our project management practices, tools and techniques, and helping clients work effectively with them.
  • Project Controls: We have many complex projects and need to manage resourcing across them carefully. We also need to ensure that we have the appropriate management information for decision making – whether it is to
    determine our ability to take on new work, feed into our cash flow and budgeting plans, or analyze project profitability.
  • Account Handling: We only employ Senior Developers who are also skilled project managers. However there is often a requirement to attend client meetings to assist in matching client and lead developer, helping to ensure that
    requirements are captured, high level business issues are understood, and
    linking requirements to solution roadmaps. It also involves acting as a proxy
    to make sure that our commitment to the client is fulfilled and so the role may also involve supporting developers in applying their chosen project management methodology.
  • Business Development: Supporting our Executive Director with winning new business through managing the development of proposals and pitches, farming and sizing new business opportunities (with new and existing clients), and actively managing ITT sources.
  • Product Management: From time-to-time we turn some of the bespoke applications that we have developed into products. When this happens we need someone to help write business cases, develop and implement marketing and brand plans and, through true customer insight, assist in steering the product development direction.

Ideally you’ll come from a project management discipline with a mix of planning, technical (at the requirements level) and client facing experience. In return, you can expect;

  • Working with some of the best people in the industry and some of the
    World’s largest companies, including Barclays, BBC, Dorling Kindersley,
    Habitat, Levi’s, Microsoft, The Discovery Channel, T-Mobile, Vodafone, and
  • Working in the heart of Hoxton, surrounded with all the bars and eateries you could possibly want or need.
  • Extremely flexible working patterns, season ticket loans, and contributory group pension and healthcare schemes.
  • A fridge with lots of soft drinks and beers, an excellent coffee machine, and an office jukebox to rival all others.
  • An opportunity to make the job what you want it to be, in an environment
    that’s open to new ideas and encourages radical thought.

For more information about the role, or to apply, please contact Chris
Stephenson on chris (at) or 020 7749 7749. CVs should be accompanied by a covering letter.

Closing date for applications is Wednesday 17th October, although applications will be reviewed as soon as they are received.


Minimal Erlang SMTP, POP3 server code

Some seven months ago, I built simple Erlang modules for generic SMTP and POP3 services. The idea is that the programmer should instantiate a service, providing callbacks for user authentication and for service-specific operations like handling deliveries, and scanning and locking mailboxes. Originally, I was planning on providing SMTP-to-AMQP and AMQP-to-POP3 gateways as part of RabbitMQ, but I haven’t had the time to seriously pursue this yet.

A snapshot of the code is available as a zip file (Update: quite outdated now! See github for the latest code), or you can browse the code online or retrieve it using git:

git clone git://

The current status of the code is:

  • SMTP deliveries from Thunderbird work
  • POP3 retrieval from Thunderbird works, but isn’t very solid, because I haven’t implemented the stateful part of mailboxes yet.
  • The SMTP implementation is somewhat loosely based on RFC 2821. It’s what you might generously call minimally conformant (improving this situation is tedious but not difficult). It doesn’t address RFC 2822 in any serious way (yet)
  • The POP3 implementation is based on RFC 1939.
  • SMTP AUTH is not yet implemented (but is not difficult)
  • I can’t recall the details (seven months!), but I think I might have skimped on something relating to POP3 UIDL.
  • Neither module has pluggable callbacks: the SMTP delivery-handler is currently io:format, and the POP3 mailbox and user authentication database are similarly hard-coded.

Patches, bugfixes, contributions, comments and feedback are all very welcome!

Update: a new post summarises changes since this post, including pluggable callbacks etc.


.NET is an endless supply of fascinating puzzles

In C, size_t is unsigned. In Java, there are no unsigned fixed-width pseudointegral types, so it can perhaps be forgiven for having an array’s length field be signed. In .NET, however, which has unsigned ints, an array’s length field is also signed. What could it possibly mean to have a length less than zero?


LShift at Agile Business Conference

The fourth Agile Business Conference will be held at the QE2 Conference Centre from 2-3 October 2007. The conference represents a great opportunity to find out more about a range of Agile development approaches. LShift is sponsoring, exhibiting, and attending the event along with other sponsors such as Borland and Fronde. Organisations such as Allied Irish Bank, the BBC and British Telecom will be discussing their experience of Agile adoption and development.

This is an exciting period for the Agile community with a rapid
acceleration in the uptake of Agile methods, both within and beyond the IT
context. The conference has been organised jointly between the DSDM Consortium – of which LShift has been a member since 2002 – and the Agile Alliance.

Please feel free to come and talk to us in the exhibition space at the event to find out more about what we think about Agile.


Erlang on Neo1973 cellphone

This evening, after fighting bitbake (in the form of the capricious “insane.bbclass” class definition) for a good few hours, I managed to get Erlang version R11B-5 running on my new cellphone.

Running the interactive erlang shell on a cellphone is pretty cool. Erlang’s built-in clustering support works fine: I’ve successfully connected an erlang node on my pc to a node on the phone using the USB ethernet support the phone provides.

The base package compiles down to a bit less than 7MB, which is a bit large. The full suite of libraries are another 22MB or so. It’s certainly possible to fine-tune the packaging process to get a smaller distribution, but for now I’m happy developing against what I have.

Update: I’ve posted my changed build scripts to OpenEmbedded’s bug tracker at bug 3014. Here’s a direct link to the tarball, if anyone would like to try it themselves.


Most exciting programming language I’ve seen in months

This [old link: ] (LTU discussion here) is the most exciting programming language implementation I’ve seen in months. Time to learn more about Forth!


How should JSON strings be represented in Erlang?

Erlang represents strings as lists of (ASCII, or possibly iso8859-1) codepoints. In this regard, it’s weakly typed – there’s no hard distinction between a string, “ABC”, and a list of small integers, [65,66,67]. For example:

Eshell V5.5.4  (abort with ^G)
1> "ABC".
2> [65,66,67].

Erlang also has a binary type, a simple vector of bytes. In the rfc4627/JSON codec I made for Erlang, I chose to use binaries to represent decoded strings, as suggested by Joe Armstrong.

All was well – until I came to implement UTF8 support after Sam Ruby got the ball rolling. Binaries will no longer work as the chosen mapping for JSON strings, since strings may contain arbitrary characters, including those with codepoints greater than 255.

It has always been the case that the ideal representation for a JSON string is an Erlang string, a list of codepoints. Binaries are really a bit of a compromise. But choosing strings-for-strings puts us straight back in a weakly-typed position: it’s possible in JSON to distinguish between “ABC” and [65,66,67], but it’s not possible to make the same distinction in Erlang. We’d need to alter the way JSON arrays are represented to compensate.

Possible solutions:

  • Map strings to lists of codepoints. Map arrays to tuples rather than lists. Objects remain {obj,[...]}.
    • Pros: Terse syntax for strings and arrays, no worse than the Unicode-ignorant mapping
    • Cons: Awkward recursion over arrays, either using a counter and the element/2 BIF, or converting to a real list
  • Map strings to binaries containing UTF-8 encoded characters. Keep arrays as lists. Objects remain {obj,[...]}.
    • Pros: Keep terse syntax for strings, with the understanding that the binaries concerned must hold UTF8-encoded text. Keeps the interface largely unchanged.
    • Cons: Codec needs to perform possibly-redundant Unicode encoding/decoding steps to ensure that the binaries hold UTF8 even if, say, UTF32 were the format to be used on the wire
  • Map strings to lists of codepoints. Map arrays to {arr,[...]}, as other JSON codecs do. Objects remain {obj,[...]}.
    • Pros: Natural operations on strings, natural operations on arrays (once you strip the outer {arr,…}).
    • Cons: Converting terms to JSON-encodable form is a pain, since you need to wrap each array in your term with the explicit marker atom.

All in all, I can’t decide which is the least distasteful option. I think I prefer the middle option, keeping strings mapped to binaries and viewing them as UTF-8 encoded text, but I really need to get some feedback on the issue.


NDocProc: Javadoc-like documentation for .NET

There are a few tools for building javadoc-like documentation for .NET code available out there on the ‘net. Unfortunately, the major contenders (e.g. NDoc, Sandcastle) suffer from a few flaws: they are variously not free (gratis), not free (libre), not cross-platform, not maintained, and/or not easy-to-use. Therefore:

Presenting: NDocProc.

Download a zip snapshot from here, or use mercurial to check out the repository:

hg clone

Update: New release available.


Closing over context still not easy in mainstream languages, Film at 11

I find it fascinating that after so many decades of support for closures, we’re still stuck in a C-style mentality of passing function-pointers that take an explicit context argument rather than a proper closure object. Witness the design of .NET’s Type.FindInterfaces method:

public virtual Type[] FindInterfaces (TypeFilter filter,
                                      Object filterCriteria);

The TypeFilter argument is a delegate. The Object argument is context that the delegate may require! This is pretty much exactly the old-school C-style way of implementing closures:

/* Yes, pretty crude translation, I know */
TypeArray find_interfaces(int (*type_filter)(Type*, void*),
                          void *argument);

Smalltalk (and Lisps) would do it in the natural way, with a block (a closure):

someType selectInterfaces: [:interface | ... ]

Lisp 1.5, complete with support for lexical closures, appeared in 1959. It’s 2007. That’s forty-eight years.




You are currently browsing the LShift Ltd. blog archives for September, 2007.



2000-14 LShift Ltd, 1st Floor, Hoxton Point, 6 Rufus Street, London, N1 6PE, UK+44 (0)20 7729 7060   Contact us