technology from back to front

Archive for June, 2009

PubSub-over-Webhooks with RabbitHub

RabbitHub is our implementation of PubSubHubBub, a straightforward pubsub layer on top of plain old HTTP POST — pubsub over Webhooks. It’s not well documented yet (understatement), but that will change.

It gives every AMQP exchange and queue hosted by a RabbitMQ broker a couple of URLs: one to use for delivering messages to the exchange or queue, and one to use to subscribe to messages forwarded on by the exchange or queue. You subscribe with a callback URL, so when messages arrive, RabbitHub POSTs them on to your callback. For example,

– is the URL for delivering messages to the “” exchange on our public test instance of RabbitMQ, and

–\_queue\_name is the URL for subscribing to messages from the (hypothetical) queue “some\_queue\_name” on the broker.

(The symmetrical …/subscribe/x/… and …/endpoint/q/… also exist.)

The PubSubHubBub protocol specifies some RESTful(ish) operations for establishing subscriptions between message sources (a.k.a “topics”) and message sinks. RabbitHub implements these operations as well as a few more for RESTfully creating and deleting exchanges and queues.

Combining RabbitHub with the AMQP protocol implemented by RabbitMQ itself and with the other adapters and gateways that form part of the RabbitMQ universe lets you send messages across different kinds of message networks — for example, our public RabbitMQ instance,, has RabbitHub running as well as the standard AMQP adapter, the rabbitmq-xmpp plugin, and a bunch of our other experimental stuff, so you can do things like this:

RabbitHub example configuration

- become XMPP friends with (the XMPP adapter gives each exchange a JID of its own)

- use PubSubHubBub to subscribe the sink to some PubSubHubBub source — perhaps one on the public Google PSHB instance. (Note how the given URL ends in “x/pshb”, meaning the “pshb” exchange — which lines up with the JID we just became XMPP friends with.)

- wait for changes to be signalled by Google’s PSHB hub to RabbitHub

- when they are, you get an XMPP IM from with the Atom XML that the hub sent out as the body

RabbitHub is content-agnostic — you don’t have to send Atom around — so the fact that Atom appears is an artifact of what Google’s public PSHB instance is mailing out, rather than anything intrinsic in pubsub-over-webhooks.

We’ve also been experimenting with using to run a PubSubHubBub endpoint in a webpage — see for instance and its associated Javascript for a simple prototype of the idea. I’m playing with building a simple PSHB hub in Javascript using the same tools.


ICFP Contest 2009

What is fast becoming a regular fixture in my diary is my entry with a few friends into the ICFP Programming Contest each year. This is a three day programming competition in which you can write in any language to solve the problems given. The competition is still in progress, though my team’s decided to stop — we’ve had enough fun for one year, and there’s only so far you can go with very little sleep and shockingly poor maths — but we’ve done much better than last year, learning from our previous mistakes… Read more…


Python Queue interface for AMQP

Here at LShift we’re often discussing RabbitMQ. We’re keen about complicated deployment scenarios, redundancy of the broker and other complex use cases. While these problems are extremely interesting, some believe they are irrelevant for a great majority of RabbittMQ users.
People keep asking how to get started with Rabbit. There are some very good sources however, understanding the AMQP abstractions requires some time.
Having that in mind I was astonished when I’ve seen this code, where Brian wraps AMQP code with a very simple Queue-like interface. This reminded me that messaging can be trivial and intuitive. In some environments, a queue is exactly what you need from messaging.

Read more…




You are currently browsing the LShift Ltd. blog archives for June, 2009.



2000-14 LShift Ltd, 1st Floor, Hoxton Point, 6 Rufus Street, London, N1 6PE, UK+44 (0)20 7729 7060   Contact us