Why is this interesting? Well, because it shows a slightly different view of the web. There’s an awful lot of hyperlinks out there, especially when you get to heavily interconnected places like Wikipedia, but in a way that’s far too much data. Instead, we’ve got some data that shows how you interacted with the web, and if we’ve got a reasonable visualisation tool, then we can learn a few things.

Enter the history crawler. Give it one or more history databases from your browsers (defaults to looking only at the local data, but it can be handed copies from your other machines as well). It then generates Graphviz graphs that show your own personal path through the web, sub-dividing them when we can’t find a history record that connects two subgraphs.

(Side note: I ended up having to use dot, fdp and neato. Dot gave the most useful input, but was most likely to crash repeatedly when I handed it something complex, and fdp and neato made reasonable backup options when that failed so we at least get something out)

So, let’s now have a look at some of the results of this.

This shows me looking for information on the height of a cassette tape (for reasons that may become clear in a later blog post if a certain item eventually turns up…). It starts with a google search for “height of cassette” and then I clicked on the result for the Wikipedia page for “Compact Cassette”, and then read a bunch of related Wikipedia pages. A couple of items are of note: firstly, the boxes marked “www.google.co.uk” and “en.wikipedia.org/wiki” – these enclose a collection of links with a common prefix, which reduces the amount of extra text needed on screen, and helps to group links. Also, for Wikipedia, and various other similar sites, it happens to give nice labels for pages! For the most part, these nice labels just fall out of the URL parsing, but there’s a couple of special cases for Google URLs as Google searches were common items in my history and the search terms get buried a bit in the rest of the URL. Overall, it’s a pretty interesting snapshot of part of my browsing history.

Often the graphs are quite small – there appears to be more disjointness in a browsing session than you’d expect, but some are larger

and some get really messy…

Overall, it was an interesting experiment, and I’ve certainly got a bit more knowledge about how long I spend digging around Wikipedia!

Perhaps less well known is the unfold, at which we are going to look today.

If you read the previous link you’d see the term “anamorphism”, described as the dual or complement or inverse of a “catamorphism”. Some folks like to talk fancy, I suppose. Unlike fold, which has a multitude of different non-Greek names (reduce, inject, and so on), I have yet to come across synonyms for unfold.

1. A seed value,
2. a function expanding a value into some structure,
3. a function transforming the seed in some manner,
4. a function telling us when we’re done.

Since I’ve been playing around with Michael Lucas-Smith and Martin Kobetic’s XTreams library on another project (about which I hope to blog in the near future), let’s use that as a starting point.

Verbs in the Smalltalk collection API follow an -ect pattern – collect, select, reject, inject, detect, etc. – so let’s follow in that grand old tradition. Given that we have to supply a next-seed and we’re-done predicate,

Seems like a good name. Oh, and XTreams uses present participles rather than verbs to emphasise that we’re working with streams of data, so let’s use

.

Name in hand, let’s go. First, we need to describe the behaviour we want to see: what should this new-fangled thing actually do?

    "Turn a number into a (descending) list of numbers."
    self assert: #(3 2 1 0)
        equals: (4
            ejecting: [:n | n - 1] "At each step, return the predecessor of the current seed."
             nextSeed: [:n | n - 1] "Count down"
             until: [:result | result = 0]) rest.

What’s that

    Object >> ejecting: expansionBlock nextSeed: seedBlock until: predicateBlock
        "Return an XTReadStream that progressively turns self into some
         structure. expansionBlock tells us how to unfold one step, seedBlock
         tells us how to get the next seed value, and predicateBlock returns
         true when the unfolding has finished."
        ^ [:out | | seed |
        seed := self.
        [predicateBlock value: seed] whileFalse:
            [out put: (Expansionblock value: seed).
            seed := seedBlock value: seed]] reading.

Note that the above looks just like a normal do-while loop, yet the result is a stream. What gives? XTreams sees that the block takes an argument, and so instantiates an

Unfold in hand, we can now do amazingly useful things like combining an unfold and a fold to transform an integer into its binary representation:

    (((42
        ejecting: [:n | n // 2]
        nextSeed: [:n | n // 2]
        until: [:n | n < 1])
            collecting: [:n | (n \\ 2) printString])
                injecting: '' into: [:l :r | r , l]) rest. "Note that we PREPEND the bit."
    "=> #('1' '01' '101' '0101' '10101' '010101')"

And that, if one wishes to amaze and confound your peers, is what we call a hylomorphism.

1. Code which is intended to be open-source but which doesn’t have any licence on it.
2. Attribution and copyright around contributions is lacking.

These two issues have bothered me with enough frequency lately that maybe it’s worth explaining in long form. One of them might be seen to be amateur copyright lawyering, at least, although I haven’t been able to find a definitive answer on the issue of contributions. The attribution part is important, in either case, and it’s linked to the copyright.

There’s a core fact about copyright which is central to both points: when producing something original which can be copyrighted, then by default you, the author, own the full copyright over that original work, and nobody can legally modify or redistribute it. All rights reserved. The Berne convention makes this international.

This has two consequences. Namely, code needs a licence and contributions should lead to attribution.

The licence is the actual legal weight behind your intent to release code as open source, which probably is your intention if you’re, say, putting it on GitHub in a public repository.

Without an adequate licence, then, because of the default, technically nobody can use what you’ve released in the way you probably meant it to be used.

The second consequence is that, if you accept significant contributions from other people, their work can’t legally be reused. A related aspect of the contribution thing is that, from a moral and ethical standpoint, credit should be given.

If you want your code to be freely modifiable or redistributable, you need to add a licence file that covers your project or your chunk of code, or whatever it is you’ve produced. If you don’t, then you’re leaving a legal obstacle in the way of someone who wants to use your code. They can’t legally modify and improve it, they can’t use it in their own projects.

Anyway, the point is, include a licence! Make it clear what your terms of use are, and avoid, from the very start, a situation where someone comes along and forks your project into some derivative work. GitHub, for example, makes it so easy to fork a project that quite quickly a house of cards can start building up with a foundation based on code that didn’t explicitly grant any right to modification; GitHub doesn’t concern itself with copyright or licensing of projects — something that other people have noticed is a slight problem, too.

Whatever you pick, it should be something legal with clear language. Clear language so that it’s understandable by human beings in a short space of time. Legal language so that it has weight and isn’t very vague. It turns out wording is important.

So what about the other side of things? Someone sends you some code they wrote which fixes a ton of bugs and implements a few features you wanted to get around to writing but hadn’t had a chance to. This is great! Open source software combined with the border-smashing power of the Internet to bring you and someone across the globe together over diffs and code review. Merge right now!

Wait, though. There are two somewhat related things which I think get overlooked too often:

1. The aforementioned copyright aspects of the author’s work.
2. Attribution.

Code someone else wrote and sent to you belongs to the author, certain circumstances notwithstanding. Why should you care? Because it carries questions with it. How come it’s being used in this project? Can someone new to the project make changes to that code freely? If the code was submitted as free software, then, again, why aren’t they given credit in a copyright notice somewhere, satisfying the licence?

By attribution, I mean that when you go to some projects, you check the software licence being used and, again, the copyright notice says that there’s only one person involved. But, if you check the number of contributors, there’s more than one person’s work. Finding projects like that is really, really easy. What happened to all those people? Why is there only one (or sometimes two) people mentioned? If the code is significant, they should almost certainly get credit. Even if it’s insignificant and not, in a legal sense, their code, even a 1-line de minimis contribution which can’t be legally owned has its value, so I think attribution is a good thing to do regardless of the size of the contribution. Completely separate from any legality, it’s rewarding to any contributor to get the acknowledgement.

So now the intellectual property side of things.

They wrote that code so by default they own the copyright. Unless they license their contribution, it technically isn’t allowed to be modified or distributed or any of that good stuff by other people who later make changes to it. But isn’t this an open source project?

The MIT licence doesn’t say anything about contributions. The GPL, through its copyleft nature, forces any derivative work (the contribution you’re merging) to be covered under the same terms as the original. Unless you’ve got something else in the mix that covers contributions or modifications, the code being sent needs to have some licence covering it.

This isn’t a new problem, by any means, and there are already different ways to handle it. Three of them are:

1. Do nothing and let it be implicit that the contribution is licensed under, well, implicit terms. Rely on the assumption that “it’s obvious what was intended.”
2. Follow a more corporate solution and get a Contributor License Agreement signed by the contributor, or, as a lightweight alternative, just state the terms in the one-off submission. This makes everything explicit.
3. Transfer the ownership of the copyright to a specific person or a group or some other entity.

If you go for the first solution, it’s vague, and with a lot of contributors, it means steadily adding more and more code from different people with different levels of understanding about software licences, and different expectations. What does the law even say here? Has it been tackled?

The second solution means having to remember to do this as a contributor, and remembering to check it’s been done as a project maintainer or developer.

The third is not only paperwork but is probably unpleasant for an open source project; contributors want to keep ownership over their modifications, they’re not employees of the developer.

An alternative fourth option to the above is to make it clear in the description of the project, maybe in the README, how contributions will be treated by default. “Any contributions will be assumed to be under the same terms as this library/application/project.” “Any contributions will be assumed to be licensed under the MIT licence.”

The Apache licence version 2 takes this approach:

Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.

Or, in short, “if you send me a patch, it is immediately Apache-licensed unless otherwise stated.”

This almost starts to feel like copyleft, where modifications are forced to have the same strict requirements. But there’s a difference between hacking on a project as part of a derivative work and submitting contributions back to an upstream project. Copyleft is about what’s gone out, whereas this is about code coming in.

(Although the licence suggests adding a copyright notice to every file, it doesn’t strictly require it. Sometimes a file shouldn’t have a copyright notice asserting ownership over them, because it doesn’t make sense.)

So if we take it as given that contributions are now properly licensed, and, say, your MIT-licensed library gets MIT-licensed contributions, there’s now a community- and legal-focused question of where to make it clear that there’s more than one person who added code. It should be made clear somewhere because it’s the polite thing to do, but also because you need to add a copyright notice somewhere covering the contributing authors.

A few options:

1. Keep a list of all contributors to a file in the file itself.
2. List every contributor in the LICENSE file or README, along with the original author.
3. Have the LICENSE file or README, in addition to prominent authors, reference a list of people in some CONTRIBUTORS file.

The first one gives a “permanent” tag on every file, but maybe it’s too intrusive or too much metawork.

The second one you might not like because it gives equal weight to every single contribution, large and small, whereas one or two people are the main developers. Still, you could be descriptive about it by tweaking the licence text.

The third one seems the cleanest. So, for example, a copyright notice might look like

Copyright © 2012, Adam Prescott and other contributors (see the CONTRIBUTORS file).

and the CONTRIBUTORS file can have some broad, plain-english description of who added what, or just simply be a list of people. Since you’re just referencing a file, CONTRIBUTORS can list those who have contributed code and those who have contributed other ideas.

While it’s by no means a novel idea, it does allow you to make it very clear that you’ve received contributions which are okay to modify and reuse, and, secondly, makes it very clear who those people are! Attribution is thus covered. As an extra bonus, if a contributor adds their own name to the CONTRIBUTORS file, then — in combination with some clear, up-front contribution guidelines — that’s a good, simple indicator that they’re sending you work under the same terms.

Let’s summarise.

If you’re posting code beyond a significant size that you expect people to use, don’t let it be locked away behind “All Rights Reserved.” Pick a licence for your code so that others can use what you’ve made without having to come to you for permission. Pick a legally-weighty licence you actually understand.

Take care with contributions, and think about giving credit.

Copyright is pretty much an endlessly deep topic, so there are all sorts of other things to think about. I’d like to have the answers to some of them. For instance, is it even legally acceptable to just say “contributions will be assumed to be under the MIT licence” unless that’s expressed in the licence? In the case of GitHub, when someone makes a public fork of a project and makes changes, their changes are immediately viewable and forkable, so if they’re to be proper about it, should they add themselves to the contributors file before anything is even contributed back upstream?

But that’s enough words, I think.

As it turns out, not too awful. I needed to make a few small modifications, and we’ve now managed to deploy a working mercurial-server under Windows.

Having said that, here’s a BIG FAT WARNING: Windows is not a supported system. We’ve gotten this guide to work ok on Windows 7 and 2003, but the permissions are still broken in various ways to do with the underlying issues in Windows’ permissions system. Unix-based systems, especially Debian and derivatives are still the main target. This is very much a “you break it, you keep both pieces” guide, but patches are welcomed!

Assuming you’ve made it through that proviso, here’s how to do it:

1. Checkout a copy of mercurial-server from mercurial (obviously!) with

   hg clone http://hg.opensource.lshift.net/mercurial-server

   and checkout the windows branch
2. Install Cygwin using the setup.exe from there. Tested against 1.7.12-1, may work with earlier versions. You’ll need the base install plus the following additional packages:

   docbook-xsl
   libxslt
   make
   mercurial
   openssh
   python
   

   Note that even if you’ve already got them installed in Windows, Python and Mercurial still need to be installed in Cygwin.
3. Open a new Cygwin terminal as Adminstrator (under Windows 7, that’s right click on the “Cygwin Terminal” start menu option and pick “Run as administrator”)
4. Run

   ssh-host-config -y
   cygrunsrv -S sshd

   to get sshd running. We need this because there’s no proper su in Cygwin (see the Cygwin FAQ for why this is)
5. Goto the mercurial-server folder and

   sh windows.sh setup-windows

   . If you had an external Python installed, this may fail. In which case try

   unset PYTHONHOME

   then try again
6. Run

   passwd hg

   and set the password for the hg user (created by setup-windows), then

   mkpasswd -l > /etc/passwd

   to add them to the Cygwin password database
7. Run

   sh windows.sh inituser-windows

   to finish setting up the hg user
8. Following the example from the main mercurial-server documentation (in that your username is jay and your machine is called spoon), but with a few differences for Cygwin, we can now get you initial access. We assume that you’ve generated a key with PuTTYgen and then copied the contents of the “public key” box into a file called my-key. Run the following, altered as appropriate for your local system:

   mkdir -p /etc/mercurial-server/keys/root/jay
   cp my-key /etc/mercurial-server/keys/root/jay/spoon
   chown hg /etc/mercurial-server/keys/root/jay/spoon
   ssh hg@localhost /usr/local/share/mercurial-server/refresh-auth

The rest of the instructions in the normal mercurial-server documentation should now work. Note that although it’s possible to add keys/access info to /etc/mercurial-server it’s much easier to do things via the hgadmin repository (and can be done from places other than the Cygwin-running server), and doing things via /etc should probably be limited to only if you mess up the setup in hgadmin.

Hope that all worked for you! We’d love to hear if anyone else ends up using this out in the field, and anything extra you’ve done to get it working on your particular system.

We were aware of a few severe problems shortly before our involvement, and during the handover meeting I asked the developer how he had fixed them. He replied that if two or more users tried to manipulate the same object at the same time, it would cause the browser code to crash as multiple conflicting change events would be sent. He therefore added a new event type that was sent out when an action started, e.g., on mouse down. This indicated that all the other browsers should lock their copies of the object in question until the action was finished.

I gently suggested that this might not work well over a network, to which he responded that in testing the lock events took less than 10ms to arrive, so the odds of two people trying to do the same thing within that period were so small as to not be worth worrying about.

Again, I very gently suggested that the benchmark was unlikely to hold across the Internet. He told me that he had tested it on several networks including over the public Internet with someone in the US and it was still good. I gave up at this point, and just noted to the client that this was something that would need attention fairly promptly.

This assertion that there was a latency of 10ms between the UK and the US is clearly false. Even if you could set-up a perfect vacuum between London and New York, it would still take 19ms for light to travel that distance. However, the numbers are irrelevant – there is latency and it must be considered when trying to build a reliable distributed system regardless of magnitude. This and seven other issues were famously covered in the “Eight Fallacies of Distributed Computing”, assembled by Peter Deutsch, James Gosling and others.

Anyone building a distributed application should have these fallacies in mind, and this paper has a good explanation of them with a very readable commentary. NodeJS made this shared browser experience extremely effective and enticingly simple to construct, but even NodeJS apps are not isolated from the effects of the infrastructure they utilise. In fact, until TCP over entangled photon pairs hits the shelves, these fallacies will be relevant regardless of your framework.

The blogosphere/twitspace is all abuzz over Bret Victor’s recent demonstration Inventing on Principle. I’d come across Bret’s work through his Up and Down the Ladder of Abstraction active essay. (Bret calls these things “reactive documents”. The concept’s a lot older, of course, and Alan Kay would call these active essays. Given my personal history, I’ll also call them “active essays”.) One of the tricks that makes Ladders of Abstraction so engaging is the ability to, in real time, tweak the model under examination. I won’t pretend to have the taste that Bret shows, and will restrict myself to one tiny piece of the UX puzzle – TangleKit.

TangleKit exists for a single purpose: to allow the simple manipulation of one-dimensional input variables, and the chance to perform arbitrary calculations when those inputs change. The setup is simple: you have a

We have six variables in this demo, so let’s just confine ourselves to the real component of the first translation. The HTML looks like this:

  <div id="moebius">

    <span data-var="f1_r" class="TKAdjustableNumber" data-min="-2" data-max="2" data-step="0.0625"></span>

  </div>

declaring the variable, its range and its resolution. The corresponding Tangle is then

  function setUpTangle() {

    var element = document.getElementById("moebius");



    var tangle = new Tangle(element, {

      initialize: function () {

        this.f1_r = 0;

        // Set up the other variables here in a similar fashion.

      },

      update: function () {

        // this.f1_i and friends are set up just like f1_r

        var f1c = complex(this.f1_r, this.f1_i);

        var f3c = complex(this.f3_r, this.f3_i);

        var f4c = complex(this.f4_r, this.f4_i);



        // Do anything else we might want to do



        // Now that this.a_r is declared and initialised we can

        // now refer to a variable "a_r" within the HTML. this.a_r

        // represents the real component of the numerator's

        // z co-efficient:

        this.a_r = f4c[0];

      }

    });

  };

We can now render our mutated “output” variables thusly:

Finally, let’s play!

Enter NAntScript. Originally from some folks elsewhere, but languishing and somewhat abandoned and in need of some work. The core idea being that you can write new tasks for NAnt using existing tasks (it is in fact also possible to write new NAnt tasks using C# or various other languages, but that’s a pretty messy way to work)

This enables you to write common sets of tasks e.g. a copy with a filter for templating, as a new task which can then be repatedly reused.

However, NAntScript had a few little flaws. It didn’t work on modern NAnt instances (luckily a small fix), there wasn’t any online documentation for it (also now fixed), and you couldn’t write functions with it, and that last one is the really fun new one I’ve added.

In the same way as the earlier taskdef let you create a custom task, funcdef lets you make a custom function. Functions are a bit different, in that they’ve got properly typed inputs and outputs, and that you’ve got to figure out a way how to get a value out without breaking NAnt syntax too badly, but that’s solvable in a similar way to how variables work in func/taskdef’s. Variables are defined as being called some “foo” and then they are inserted any time you enter the magic text “__foo__”, and the return works in a similar way, defining a magic return variable called “__$return__” which must be set at some point during the internals of a function. Here’s an example of that. Source as usual is on github, and I’ve even added some precompiled binaries for download.

Recall that a complex number may be represented by a pair of real numbers. As such, representing a set of complex numbers – a region – is straightforward: an Argand diagram does the job just great. (Think of a Cartesian plane, and label the x-axis “real” and the y-axis “imaginary”.) Usually we just call this the z-plane, and the plane to which we map – the range – we call the w-plane.

What about a function though? The obvious visualisation is a bit hard: since visually representing a complex number requires two numbers, we’d need to use four dimensions to visual a complex function of one variable. Sadly, I left my 4-dimensional graph paper in my other bag.

Several other possibilities spring to mind: split the mapped values into their real and imaginary components and draw each separately. Since we now only have three dimensions left (the domain’s two, and one for the range), things are a bit easier. Typically we map the third dimension to a colour, giving something like a contour map. MathWorld is filled with examples.

It’s also easy to show how the shape of some region changes. Define a region in the z-plane – a disk, say – and draw in the w-plane the boundary of the region. And we can take it a step further: we can draw the mutations of a grid of lines. Right, enough maths. Let’s draw!

The basic idea in d3 is simple: you have some data in an array, you have a selection of nodes in an array, and you want to map the one onto the other. Today, we’re going to use SVG

  d3.select("#z-plane").append("svg")

    .attr("height", 300)

    .attr("width", 300)

    .append("g")

    .attr("fill", "none")

    .attr("stroke-width", 2)

    .attr("stroke", "black");

Next, let’s make some data. We want some lines forming a grid, and those lines will be polylines with a large number of points on the lines. (Pretty pictures need lots of points for nice smooth curves.)

  function hlines(gridResolution, pointResolution) {

  return gridResolution.map(function(i) {

      return pointResolution.map(function(j) {

        return complex(j, i);

      });

    });

  }

  

  function vlines(gridResolution, pointResolution) {

    return gridResolution.map(function(i) {

      return pointResolution.map(function(j) {

        return complex(i, j);

      });

    });

  }

  

  function makeGrid(gridResolution, pointResolution) {

    return hlines(gridResolution, pointResolution)

             .concat(vlines(gridResolution, pointResolution));

  }

  

  var min_x = -2.5, max_x = 2.5;

  var interline_distance = 0.2;

  var interpoint_distance = 0.05;

  var grid = d3.range(min_x, max_x + (interline_distance / 2), interline_distance);

  var pointsOnLine = d3.range(min_x, max_x + (interpoint_distance / 2), interpoint_distance);



  var zlines = makeGrid(grid, pointsOnLine);

  var wlines = makeGrid(grid, pointsOnLine);

We map the data onto the SVG elements. Now we don’t have any SVG elements at the moment. That’s OK. Thanks to

  function makeLines(planeDiv, arr) {

    planeDiv.selectAll(".line")

      .data(arr)               // Bind to all the line class elements

      .enter().append("path")  // and add path.line element as necessary

        .attr("class", "line")

        .attr("d", lineMaker);

  }

But how do we actually display the data? d3’s scales map our domains – two continuous ranges of values from -2 to 2 – to our ranges – 300 pixel long sides:

  var x = d3.scale.linear()

    .domain([min_x, max_x])

    .range([0, h]);

  var y = d3.scale.linear()

    .domain([min_x, max_x])

    .range([h, 0]); // Invert the scale so the origin is in the BOTTOM left corner

  var lineMaker = d3.svg.line()

      .x(function(d) { return x(d[0]); })  // How we get the x-coord

      .y(function(d) { return y(d[1]); }); // How we get the y-coord

When we press the “Go” button, we fire off a click event:

  d3.select("#go-button").on("click", function() {

    // Recreate our Moebius transformation

    f = memoiseF();

  

    // And warn if our (yes, global) variables indicate that we do not, in

    // fact, have a Moebius transformation.

    if (czero(csub(cmult(a, d), cmult(b, c)))) {

      alert("Not a Moebius transform!\nMerely a constant function f(z) = " + JSON.stringify(f(complex(1, 0))));

    }

  

    // Mutate the w-plane data in place so d3 can automatically

    // pick up the changes.

    var lastw = [0,0];

    for (var lineI = 0; lineI < wlines.length; lineI++) {

      for (var pointI = 0; pointI < wlines[lineI].length; pointI++) {

        w = f(zlines[lineI][pointI]);

        if (w == Inf) {

          // If f(z) = Inf then we pretend that Inf is really a point

          // outside of visual range that's on the ray drawn from

          // the origin through lastw. Think of the resulting point

          // as being the closest point to lastw on a circle centred

          // on the origin and of infinite radius.

  

          // Multiplying by n + 0i (n a real) means scaling without

          // rotating.

          w = cmult(lastw, complex(100, 0));

        }

        wlines[lineI][pointI] = w;

        lastw = w;

      }

    }

    drawLines(wplane, wlines);

  });

And lastly, refreshing our view is trivial:

  function drawLines(planeDiv, arr) {

    planeDiv.selectAll(".line")

      .transition()   // Change the view of the data ...

      .ease("linear") // ... and make it nice and smooth.

      .attr("d", lineMaker);

  }

In short, d3 is a neat, powerful tool for visualising data. It’s highly extensible, well-supported, and can do pretty much anything you might want to out of the box, from bullet charts to Voronoi diagrams.

Well, that’s what it looks like at first glance. Once you start digging around, it turns out that the panel you’re now seeing is superficially similar to the old one, but it’s built in a very different way under the hood, and that makes writing applets for it a somewhat different problem. Primary among the changes for the panel is the removal of Bonobo in favour of D-Bus, so we’re going to have to build the services needed for applets in a new way.

Now, if you start looking around the web for guides to making Gnome panel applets, you’ll find two things. a) lots of guides of how to do this in a 2.x environment (e.g. this) but little/no 3.x guides, because of b) that applets are officially deprecated as of 3.x (that, and 2.x has been out since 2002 v.s. 2011 for the 3.0 release). The reason they’re officially deprecated is that we’re all meant to be using the Shell now, which is a very different thing, and so needs to do things in a completely different way. However, I (and I suspect a fair chunk of the world) will be sticking with non-Shell environments for a while yet, so there’s still a need for 3.x panel applets.

Earlier on I noted how the 2.x and 3.x panels were very different things, but despite the major interface changes, your core applet is still going to be the same. In both cases, some form of messaging bus (Bonobo, D-Bus) provides information about what applets are available, and then invokes the launchers for them on demand. For either 2.x or 3.x you’re going to be writing a whole bunch of messaging system boilerplate, when what you really want to be doing is actually writing your own applet!

I’ve therefore written a little boilerplate generator that makes applets that work on both 2.x and 3.x panels. Using Genshi, it generates the Bonobo server files, D-Bus service files, a pair of factory helpers that build either a 2.x or 3.x applet, and then a simple applet stub that you can expand that looks like the following:

It’ll also generate a bunch of Debian package files so you can package up your applet right away. It’ll work straight out of the box, with a little label saying “It works!”, but you can then customise just the applet stub file and it’ll happily continue to work with both 2.x and 3.x. One potential issue is making sure that you’re importing from gi.repository for 3.x and then falling back to the relevant individual modules for 2.x as shown at the top of the applet stub, but otherwise it should be fairly simple to build your applet of choice.

As the interface files need to be hardcoded to a particular path for the applet, the best option for further development is to install the generated package, then make the main applet (/usr/lib/gnome-applets/<applet-name>Applet.py) a symlink to a version of the file that you can edit. If you then add the applet to your panel, the python process for the factory (called <applet-name>-factory{2,3}.py depending on your panel version) can be killed every time you want to refresh the code (don’t forget to answer yes when the panel asks if you want to reload the applet that just died!). Unfortunately given the process model for panel applets, there aren’t any easier ways to do development, but this works fairly well.

The DSL describes domain objects in a language-agnostic fashion. For example:

  in_namespace('Outer') {

    in_namespace('Inner') {

      define('An Inner Object') {

        field 'name', :string

        field 'value', :integer

      }

    }

    define('An Outer Object') {

      field 'name', :string

      field 'wrapping', 'Inner.An Inner Object'

    }

  }

defines a pair of objects, one designed to wrap around the other. Using this language, one can generate data objects in some language of your choice: Smalltalk, perhaps, or Haskell.

The DSL has one extra bit: because the language describes a wire contract between two services, it’s sensible to version the entities, so that one can quickly see that two services have gotten out of sync: one or the other needs upgrading, for instance. So our domain model might look like this:

  in_namespace('Outer', :version => "1.0") {

    in_namespace('Inner') {

      define('An Inner Object') {

        field 'name', :string

        field 'value', :integer

      }

    }

    define('An Outer Object', :version => "2.0") {

      field 'name', :string

      field 'wrapping', 'Inner.An Inner Object'

    }

  }

This works pretty well: the wrapping object has version “2.0″ while the wrapped object inherits its version – “1.0″ – from its namespace. Imagine that you have some namespace with a large number of entities. It’s very easy to accidentally upgrade all objects in a namespace when you only meant to upgrade some. So the per-namespace versioning needed to disappear, forcing the user to explicitly express all versions on a per-definition basis. Clearly, in a large model, that would be an onerous task. However, this is still plain Ruby, and there just happens to be a Ruby->AST translator lying around, so it makes sense to manipulate the AST directly, and not mess around with crazy nonsense like trying to use regular expressions or similar. And given that I just happen to have a Ruby zipper library capable of traversing arbitrary hierarchical structures, a plan forms…

Getting the AST is simplicity itself:

  require 'rubygems'

  require 'sexp'

  require 'parse_tree'

  require 'zipr/zipper'





  filename = "my-model.rb"

  file = File.read(filename)

  sexp = Sexp.from_array(ParseTree.new(false).parse_tree_for_string(file, filename).first)

  # I use ParseTree here, but really you want to use ruby_parser:

  # it works in Ruby 1.9, for starters.

which gives us a lovely S-expression:

  s(:iter,

   s(:fcall,

    :in_namespace,

    s(:array, s(:str, "Outer"), s(:hash, s(:lit, :version), s(:str, "1.0")))),

   nil,

   s(:block,

    s(:iter,

     s(:fcall, :in_namespace, s(:array, s(:str, "Inner"))),

     nil,

     s(:iter,

      s(:fcall, :define, s(:array, s(:str, "An Inner Object"))),

      nil,

      s(:block,

       s(:fcall, :field, s(:array, s(:str, "name"), s(:lit, :string))),

       s(:fcall, :field, s(:array, s(:str, "value"), s(:lit, :integer)))))),

    s(:iter,

     s(:fcall,

      :define,

      s(:array,

       s(:str, "An Outer Object"),

       s(:hash, s(:lit, :version), s(:str, "2.0")))),

     nil,

     s(:block,

      s(:fcall, :field, s(:array, s(:str, "name"), s(:lit, :string))),

      s(:fcall,

       :field,

       s(:array, s(:str, "wrapping"), s(:str, "Inner.An Inner Object")))))))

All that remains is (a) transforming the AST and (b) writing the AST out again as Ruby source. The latter’s pretty boring, so let’s just stick with the former. We need to do two things: select certain nodes, and transform them. So given some helper methods added to

  class Zipr::Zipper

    def version_all_objects(default_version)

      current_version = default_version



      # Recall that Zipper's map returns a structure of the same

      # shape as that over which we iterate (as opposed to

      # Enumerable's map, which returns a list of mapped things).

      map {|n|

        if n.kind_of?(Enumerable) then

          if (n.first == :fcall) and (n[1] == :in_namespace)

            # Possibly change current_version here

            # Remove the per-namespace version, if present

            if (n.has_version) then

              current_version = n.version_from_ns

              next n.strip_version_from_ns

            end

          elsif (n.first == :fcall) and (n[1] == :define)

            # Inject the version, if there isn't one already

            if (not n.has_version) then

              next n.add_version_to_object(current_version)

            end

          end

        end

        n

      }

    end

  

    puts SexpWriter.new.visit(version_all_objects(z).root)

which yields the desired, altered, model definition:

  in_namespace('Outer') {

    in_namespace('Inner') {

      define('An Inner Object', :version => "1.0") {

        field 'name', :string

        field 'value', :integer

      }

    }

    define('An Outer Object', :version => "2.0") {

      field 'name', :string

      field 'wrapping', 'Inner.An Inner Object'

    }

  }

Addendum: what a context’s path means

In the process of writing the above, I stumbled across some bugs in the zipr library. First, I realised that a traversal of a complicated tree would have multiple downs and ups. That caused a problem because originally “rooting” the zipper – committing all the edits to build your new structure – would detect that an edit had occured and invoke a special “root a changed structure” path. If one went up after an edit, this “something has changed” got lost. But the second issue turned out to be another instance of the same basic problem.

So we know a zipper has two parts, namely a one-hole context, and a value to plug that hole. We also know that a one-hole context is made up of a few parts: nodes to the left of the current focus point, nodes to the right, and so on. One of these pieces of information is the path from the current focus back to the root of the structure – the trail of breadcrumbs, as Learn You a Haskell calls it. Suppose we’ve just edited a node, and our path consists of a sequence of nodes a₀, a₁, …, a_n. If we move up, to node a_n, the context around that node has not changed. If we root the structure, the rooting process won’t know that something’s changed, and we’ve lost our edit. Suppose instead we move to the left or right. Our parent node is still a_n, but note that that path does not include the changed node. Again, rooting the structure does not pass through any context that says “hey, wait a minute, something’s changed”, and again our edit is lost!