technology from back to front

Archive for September, 2009

Erlang OTP Boot Files for Fun and Profit

As part of some new Rabbit features, we’ve been investigating the use of boot files to start multiple applications without requiring us to build our own application loader. After some initial experimentation, we built a fairly simple utility that would generate a .rel file, then request the Erlang toolchain to compile this and generate .script and .boot files. In doing this, we learnt a great deal about the internals of OTP startup sequences.

As a result of the experimentation, we found a number of interesting properties. Firstly, we discovered that if you go down the path of using boot files you should use Supervisors on all of your top-level applications. Whilst this is generally considered good practice in all Erlang applications, it is even more important when these applications are listed in the boot file, since if an application listed in the boot file crashes at any point the entire Erlang VM will be halted! This behaviour certainly piqued a number of people’s interests – after some consideration we agreed that it is a sensible enough behaviour, since if you instructed the VM to start with a given set of applications, then any of them crashing out probably constitutes a fairly major failure.

Through some investigation and reading of the Erlang documentation, we learnt this wasn’t actually a boot file specific behaviour. Instead, it was because applications listed in the boot file are actually started as permanent (instead of the default of temporary). The effect of marking an application as permanent is that if it crashes, the Erlang kernel will shutdown as a result. Those wishing to try this at home can simulate it with the use of

application:start(app_name, permanent)

and some rather adventurous use of the Erlang toolbar.

After the initial implementation, a number of other issues were discovered. Currently, Rabbit performs a number of “VM-configuration” tasks during the rabbit:start method. Unfortunately, using the boot file bypassed (or in the very least de-prioritised)  these operations. So investigation began into finding a way to configure the VM running Rabbit before applications were started.

Somewhat expectedly, the -eval flag is not evaluated until after the boot sequence is completed. Applications run via “-s” flags are similarly delayed until after the boot sequence has finished. So we began to dig a little deeper.

Anyone who has experimented with .boot files will have noticed the boot file isn’t plain text. However, the boot file is actually just a binary form of the associated .script file. Inside the .script file, following the various path-setup commands, are a series of MFA formatted commands, looking something like:

{apply,{application,start_boot,[kernel,permanent]}},
{apply,{application,start_boot,[stdlib,permanent]}},
{apply,{application,start_boot,[mnesia,permanent]}},
{apply,{application,start_boot,[sasl,permanent]}},
{apply,{application,start_boot,[os_mon,permanent]}},
{apply,{application,start_boot,[rabbit,permanent]}}

This opened up a whole new set of options. We tried a few different techniques to get the behaviour we were after. Currently, we’ve settled on inserting an apply line like {apply,rabbit,prepare,[]}} after the stdlib application startup. This allows us to do custom VM initialisation before the non-kernel applications are started. (We decided it would be wise not to do any application specific changes to the VM before the kernel or stdlib are started).

Until starting down this path, I certainly didn’t know there were so many possibilities for customising the Erlang boot sequence. Hopefully others might be able to find interesting things to do too!

by
paulj
on
30/09/09

Grouping and collapsing in WireIt

I have recently been modifying the WireIt code to allow collapsing of multiple containers down into 1 composite container.
A quick summary of WireIt (from their site)

WireIt is an open-source javascript library to create web wirable interfaces for dataflow applications, visual programming languages, graphical modeling, or graph editors.

I got started on this when I was following on from Jonathan Lister’s work on using WireIt to create a non-technical user interface for Rabbit Streams. It was soon apparent that if you wanted to use this in practice you would end up with huge graphs with many containers and many wires going all over the place. Once above about 10 containers or so with a couple of wires per container it gets hard to see what is happening. To solve this problem some kind of abstraction is needed, in this case collapsing multiple containers down into 1 composite container.

After only a few containers things get hairyShrink down the 3 groups of containers for a cleaner graph

A collapsed group of containers essentially boils down to a single container representing a sub graph. So I started by looking at the JSON object produced when you save a complete WireIt graph and also how that JSON is then expanded into the complete graph on loading. Once identified it was relatively easy to implement those function for only a sub set of the whole graph.

The next challenge was maintaining the wires across a collapse/expand. The first step was to show the terminals from grouped containers on the composite container. Naming conflicts had to be handled (in the case of two duplicate internal terminals from different containers) which led to the creation of a map from external Terminals to internal Containers and their terminals. With this map I could iterate over the wires attached to terminals in the group and create copies of those wires going to the external terminals. Then on expansion an wires connected to the composite container were again mapped back to their counter parts. The same was done for fields.

In combination with the ability to manually select which fields and terminals are visible and manually set their names you can easily represent encapsulation.

I have made a fork of the WireIt GitHub repository which can be found here. I also have a small readme and example of the editor here.

by
James Uther
on
10/09/09

HTML email from Squeak using Seaside

Recently, as part of a Seaside-based application running within Squeak, I wanted to send HTML-formatted notification emails when certain things happened within the application.

It turns out that Squeak has a built-in SMTP client library, which with a small amount of glue can be used with Seaside’s HTML renderer to send HTML formatted emails using code similar to that used when rendering Seaside website components.

sendHtmlEmailTo: toEmailAddressString
  from: fromEmailAddressString
  subject: subjectString
  with: aBlock
 | m b bodyHtml |
    m := MailMessage empty.
 m setField: 'from' toString: fromEmailAddressString.
  m setField: 'to' toString: toEmailAddressString.
  m setField: 'subject' toString: subjectString.
    m setField: 'content-type' toString: 'text/html'.
   
    b := WAHtmlBuilder new.
 b canvasClass: WARenderCanvas.
  b rootClass: WAHtmlRoot.
    bodyHtml := b render: aBlock.

   m body: (MIMEDocument contentType: 'text/html' content: bodyHtml).
    SMTPClient deliverMailFrom: m from
               to: {m to}
               text: m asSendableText
               usingServer: 'YOUR.SMTP.SERVER.EXAMPLE.COM'.

The aBlock argument should be like the body of a WAComponent’s renderContentOn: method. Here’s an example:

whateverObjectYouInstalledTheMethodOn
  sendHtmlEmailTo: 'target@example.com'
  from: 'source@example.org'
  subject: 'Hello, world'
  with: [:html |
    html heading level3 with: 'This is a heading'.
    html paragraph with: 'Hi there!']
by
tonyg
on
08/09/09

Search

Categories

You are currently browsing the LShift Ltd. blog archives for September, 2009.

Feeds

Archives

2000-14 LShift Ltd, 1st Floor, Hoxton Point, 6 Rufus Street, London, N1 6PE, UK+44 (0)20 7729 7060   Contact us