I’ve just released Snarl, a Growl-like notification system for Squeak. To use it,

Snarl label: 'Something happened'
      body: 'What could it have been?'

I’ve recorded a quick demo:

(It’s pretty blurry, so I’ve uploaded it to vimeo too, but it’s still in the queue for conversion; when it’s converted, it’ll be here.)

The code is three classes: one tiny convenience class, Snarl; one TextMorph subclass, which does almost all the work; and one helper TextAttribute subclass, for fading out coloured text along with the rest of each notification. In total, it’s 205 lines of text, including documentation.

Those guys at Arup are taking this seriously and unlike many others who postulate in this area they have the clout and the commercial imperative to influence decisions and make changes. Interesting times ahead…

Tokyo Cabinet is a rather excellent key-value store, with the ability to write to disk in a sane way (i.e. not just repeatedly dumping the same data over and over again), operate in bounded memory, and go really fast. I like it a lot, and there’s a likelihood that there’ll be a RabbitMQ plugin fairly soon that’ll use Tokyo Cabinet to improve the new persister yet further. Toke is an Erlang linked-in driver that allows you to use Tokyo Cabinet from Erlang. Read more…

Today I was lucky enough to give a talk at the Skills Matter Functional Programming Exchange. I talked about resource management in RabbitMQ and how we’re improving this in upcoming versions of RabbitMQ. All the sessions were videotaped and it would seem that a podcast will be going up shortly. In the mean time you can have a look at the slides if you want to.

The attendance was really good and the talks well received. There was a good range of talks, from some very practical and pragmatic such as my own, to slightly more theoretical talks. It was great to see Haskell, Erlang and F# being discussed outside of a purely academic setting and great to see so many companies and organisations getting really interested in functional programming and coming along to see how other people were making the most of it.

The Park Bench session was also good fun, with a good range of questions and experience being demonstrated by all. A good, fun atmosphere, and I’m sure all enjoyed the day.

How do you test a Java application that uses the current date or time as the basis for a calculation? I’ve seen a couple of ways:

1. Make sure all date/time operations use a service class to get the current date, which is then mocked
2. Adjust the system clock

Option 1) is a PITA for an existing codebase, or if any 3rd party libraries do date processing. Option 2) rules out any sensible automation, especially on a machine used for anything else.

Another option is to mock system date related classes. JMockit is a mock testing framework that uses Java 5 instrumentation to allow bytecode to be modified at runtime. Using this, known date values can be inserted into tests with no interference with production code or rewriting of legacy code. If Java 6 is available, it is possible to go further and mock the System class itself.

Read more…

mercurial-server is an official Debian package! Right now it’s only in the “unstable” distribution, but all being well it will slowly percolate forward, first into “testing”, then eventually into the stable distributions of not only Debian but Ubuntu and other Debian-based systems.

Getting it into Debian was quite a long and strange process; the care that Debian takes over package quality puts quite a burden on individual developers in order to minimize the burden on overworked Debian staff. I’ll talk through the steps I took here, eliding over any wrong turns of course, in the hope that the way I did it might be useful to others. Read more…

The new persister that is being developed for RabbitMQ is nearing completion and is currently working its way through code review and QA. It’s being pretty thoroughly tested and generally stressed to see what could go wrong. One of the issues that we’ve come across in the past has to do with Erlang’s garbage collector: indeed there’s code in at least one area of RabbitMQ written in a specific (and non-obvious) way in order to work around issues with Erlang’s garbage collection of binary data.

We had noticed in the release notes for Erlang R13B03, that it mentions improvements to the garbage collector, and today when testing with both R13B02 and R13B03, we noticed substantial improvements with R13B03. The new persister is able to send out to disk partial queues. Thus a queue can have a mix of messages - some just in RAM, some just on disk, and some somewhere in between. This is separate from whether or not a message is marked persistent. The proportion pushed out to disk varies smoothly with the amount of RAM left available to Erlang: the idea is to avoid flooding the disk with enormous amounts of write requests which would potentially stall the queue, and cause blockages elsewhere in RabbitMQ.

The test I’d written used the Erlang experimental client. It had one channel, it created a queue, consumed from the queue, set QoS prefetch count to 10, and then went into a loop. In this loop, it would publish two 1KB messages, then receive 1 message, and acknowledge it. This way the queue would always grow, and memory would be fairly fragmented (the gap from the head of the queue to the tail of the queue would increase steadily as the head is moving forwards at twice the rate of the tail). With no memory limit, I saw the following (I manually killed this after the queue grew to just over 350,000 messages long (which means 700,000 publishes, and 350,000 acknowledgements)):

Note that for R13B03, the garbage collector is much more active, and in general memory usage is certainly more fine-grained. In this test, all the messages were always in RAM, no messages were pushed out to disk. Flat-size refers to the value returned by pushing the queue state through erts-debug:flat-size/1 which returns the amount of memory used by the data structure.

Next, I imposed a limit of about 200MB and did the same test. With R13B02, it got stuck after just over 260,000 messages: it was no longer able to reclaim any further space, and so flow-control kicked in and stopped the publisher, game over. With R13B03 it soldiered merrily on - I ended up manually killing it somewhere past the 1million message mark as I was getting bored. It’s also very clear to see how with R13B03, it successfully kicks down to pushing all the messages out to disk (which is why the size of the state suddenly gets very small - the memory growth from there on is due to an ets table). That’s certainly still possible with R13B02, and I have seen that happen, but there’s much greater risk, as seen here, of it getting stuck before that happens.

In short, the garbage collector in R13B03 seems a solid improvement. Even if you’re not using the experimental new persister, I suspect you’ll gain from upgrading to R13B03. And yes, that really is 1-million 1KB messages successfully sent into a queue using under 200MB of RAM.

RabbitMQ is a becoming decent product, but it shares some of the common problems of young software - for example, beginners have a hard time understanding what happens under the hood. Don’t get me wrong, Rabbit generally works perfectly as a black-box. But at some point, when things go wrong or when Rabbit needs to be added to a monitoring infrastructure, it becomes necessary to understand more about the internals. This is when things get difficult.

I was wondering how we could address this issue and decided that Rabbit should serve a simple http status page similar to status pages that are known from Apache or Haproxy projects.

Read more…

RabbitMQ Streams is our data streams management system that allows arbitrary routing, transforming, and merging of messages. We used a “quick and dirty” test framework during development to check there were no major performance issues, but we needed to improve on this to test the real configurations used by the BBC Feeds Hub.

Read more…

LShift have used the EpiServer CMS on several customer projects and it generally does most things you would want to do with a CMS in a simple way. EpiServer is a .Net based CMS and if you understand ASP.NET templated pages and templated controls it is very straightforward with a minimal learning curve.

One challenge I faced on a recent project was to implement a particular HTML navigation design using EpiServer. The HTML design called for the navigation to be rendered as nested HTML lists with the current section of the site annotated with a particular class.

For example if you were looking at “Tasty Fish” in the “Cat Food” section of the site the HTML should look something like this:

<ul>
    <li>Dog Food
        <ul>
            <li>Meaty Bones</li>
        </ul>
    </li>
    <li class="selected">Cat Food
        <ul>
            <li>Tasty Fish</li>
        </ul>
    </li>
</ul>

On initial inspection the EpiServer CMS appears to have two controls that may help, the EpiServer:MenuList and the EpiServer:PageTree. I first attempted to use the EpiServer:MenuList, this allowed me to do this:

   <ul>
        <li>Dog Food</li>
        <li>Cat Food</li>
    </ul>
    <ul>
        <li class="selected">Tasty Fish</li>
    </ul>

This isn’t quite what the design required, the complete site navigation tree needed to be rendered since CSS was being used to show and hide menus in response to mouse rollovers.

So for attempt two I tried the EpiServer:PageTree component; this component is designed to render a whole tree of pages so it should be an appropriate solution. It is a very flexible component and provides lots of templates for customising the layout based upon the state of the tree. This is what I ended up with:

   <ul>
        <li>Dog Food
            <ul>
                <li>Meaty Bones</li>
            </ul>
        </li>
        <li>Cat Food
            <ul>
                <li class="selected">Tasty Fish</li>  <!-- OH NO THIS IS WRONG -->
            </ul>
        </li>
    </ul>

This was very close! However it didn’t meet the design requirement; the top level item that contained the current page needed to be tagged with the CSS class, not the item corresponding to the current page. There didn’t seem to be an easy way to achieve this with the EPiServer components.

I decided I probably need some type of custom control, I then proceeded to write three implementations of a navigation control moving from sinful generation of HTML in a code behind, through my own templated control until arriving at the obvious solution using the asp:ListView control and a simple code behind. This was a nice solution because it uses a standard ASP.NET component in a standard way, the complication of tagging the selected top level item could be hidden away in a small code behind, and the markup was completely under the control of the HTML developer.

The navigation section of the ASP page looked like this:

   <asp:ListView ID="Level1" runat="server" ItemPlaceHolderID="Level1Item">
        <LayoutTemplate>
            <ul><asp:PlaceHolder ID="Level1Item" runat="server"/></ul>
        </LayoutTemplate>
        <ItemTemplate>
            <li class='<%# ((Boolean)Eval("Selected")) ? "selected" : "" %>'><%# Eval("Name") %>
                <asp:ListView ID="Level2" runat="server" ItemPlaceHolderID="Level2Item">
                    <LayoutTemplate>
                        <ul><asp:PlaceHolder ID="Level2Item" runat="server"/></ul>
                    </LayoutTemplate>
                    <ItemTemplate>
                        <li><%# Eval("Name") %>
                    </ItemTemplate>
                </asp:ListView>
            </li>
        </ItemTemplate>
    </asp:ListView>

This is a straightforward usage of nested ListViews and ASP data binding expressions, all of the markup is visible and it can be explained to an HTML developer in a short amount of time. New navigation levels can be added in exactly the same way that the Level 2 navigation was added to the Level1 navigation. The ternary operator within the data binding expression, class='<%# ((Boolean)Eval("Selected")) ? "selected" : "" %>', determines if the navigation item is selected, this is a standard mechanism for conditional rendering with ASP.NET data bound controls.

This was combined with a page behind like this:

   protected override void OnLoad(System.EventArgs e)
    {
        base.OnLoad(e);

        Level1.DataSource = BuildMenuItems();
        Level1.DataBind();
    }

    private List<MenuItem> BuildMenuItems()
    {
        List<MenuItem> menuItems = new List<MenuItem>();

        PageData homePage = GetPage(PageReference.StartPage);
        foreach(PageData child in GetChildren(homePage.PageLink))
        {
            if(child.VisibleInMenu)
            {
                MenuItem item = CreateMenuItem(child, true);
                item.Selected = findPage(CurrentPage.PageGuid, child);
                menuItems.Add(item);
            }
        }

        return menuItems;
    }

    private MenuItem CreateMenuItem(PageData page, Boolean includeChildren)
    {
        MenuItem item = new MenuItem(page.PageName);
        item.Url = page.LinkURL;

        if (includeChildren)
        {
            PageDataCollection children = GetChildren(page.PageLink);
            foreach (PageData child in children)
            {
                if (child.VisibleInMenu)
                {
                    item.Children.Add(CreateMenuItem(child, true));
                }
            }
        }

        return item;
    }

    private Boolean findPage(Guid id, PageData parent)
    {
        if (id == parent.PageGuid) return true;

        foreach (PageData page in GetChildren(parent.PageLink))
        {
            if (page.PageGuid == id)
            {
                return true;
            }
            if(findPage(id, page))
            {
                return true;
            }
        }

        return false;
    }

With a helper class MenuItem defined like this:

public class MenuItem
{
    public MenuItem(String name)
    {
        this.Name = name;
    }

    public String Name { get; set; }
    public String Url { get; set; }
    public Boolean Selected { get; set; }
    private List<MenuItem> children = new List<MenuItem>();
    public List<MenuItem> Children {
        get
        {
            return children;
        }
        set
        {
            children = value;
        }

    }

}

The page behind creates MenuItem instances for each page in the navigation. The top level item gets tagged as selected only if the current page is one of its children. This is a reasonable amount of code to write but it was the smallest solution that solved the problem and made the HTML obvious and available for modification by HTML developers.