Does Clinton beat Obama?
Continue Reading 3 comments May 30th, 2008 Paul Crowley
Archive for May, 2008
Managing servers with Puppet
So, I’ve been working on this project recently. In this project there’s no use of version control - in fact, we don’t even have staging or development environments. All changes were just made to the live server, in an ad hoc way, by a variety of people. And inevitably we’ve ended up in a situation where we don’t fully understand what we have, and we’re scared to change anything in case something else breaks.
Sounds awful, right? Chances are that you’ve been working on this project too. Of course I’m referring to system administration. (In fact I’m referring to Unix system administration since we don’t have many Windows servers; the rest of this post will be rather Unix-centric.)
Clearly this situation is not right. Over the years we’ve made a few attempts to improve things - the first attempt consisted of putting /etc in version control. This is clearly a step forward - you can see who’s changed what, when - but there are still several problems. For a start, anything that happens outside /etc is not recorded - so we don’t know which packages are installed or if any changes have been made to the rest of the system. For another thing, there’s no mechanism for abstraction - if you need to make one change to all your servers there’s no sensible way to express that.
So the second attempt consists of writing scripts to build servers, and putting them in version control instead. This is another step forward - look ma, we can do package management! We can do anything! There are still problems though. We can’t just write a script which runs once to build the server; things change and we need to use the same mechanism for maintenance. So although it’s a good feeling to be programming again, we don’t really want to use an imperative language (or even really a functional one) because it’s a pain to make our scripts idempotent (and worse still, to test that they are). And while abstraction is possible in this environment, it’s not exactly convenient.
So we arrive at our third attempt to fix things, and I have a good feeling about this one. I’ve recently been starting to use Puppet, and I think it solves all the problems mentioned so far, as well as some others. It allows all your system administration to be done in one place and automatically replicated out to all your servers. It uses a declarative language that’s designed for idempotence - you describe where you want to be, not how to get there. Abstraction between servers is really easy and quite powerful.
I’ve been going around the office with a messianic glint in my eye for the last couple of days, so I’ll try really hard to think about the negatives. The language has some magic quoting rules I’m not entirely happy with. The project is new-ish and the documentation could be better (although it could certainly be a lot worse too). But that’s about it.
I will say this though: after only a couple of days of work, I’ve got to a state where we can roll out some common foundations to all our servers (use LDAP authentication, use our Debian mirror, automatically apply security updates), and where a couple of our servers are entirely Puppetised - it’s a one liner to recreate them from our Mercurial repository.
That’s a really nice place to be. And you know what? It wasn’t even that hard.
Add comment May 27th, 2008 simon
Late-binding with Erlang
Upon browsing the source to the excellent MochiWeb, I came across a call to a function that, when I looked, wasn’t defined anywhere. This, it turns out, was a clue: Erlang has undocumented syntactic support for late-bound method dispatch, i.e. lightweight object-oriented programming!
The following example, myclass.erl, is a parameterized module, a feature that arrived undocumented in a recent Erlang release. Parameterized modules are explored on the ‘net here and here. (The latter link is to a presentation that also covers an even more experimental module-based inheritance mechanism.)
-module(myclass, [Instvar1, Instvar2]). -export([getInstvar1/0, getInstvar2/0]). getInstvar1() -> Instvar1. getInstvar2() -> Instvar2.
“Instances” of the “class” called myclass can be created with myclass:new(A, B) (which is automatically provided by the compiler, and does not appear in the source code), where A and B become values for the variables Instvar1 and Instvar2, which are implicitly scoped across the entirety of the myclass module body, available to all functions defined within it.
The result of a call to a new method is a simple tuple, much like a record, with the module name in the first position, and the instance variable values in order following it.
Eshell V5.6 (abort with ^G)
1> Handle = myclass:new(123, 234).
{myclass,123,234}
2> Handle:getInstvar1().
123
3> Handle:getInstvar2().
234
While this looks really similar to OO dispatch in other languages, it’s actually an extension to Erlang’s regular function call syntax, and works with other variations on that syntax, too:
4> {myclass,123,234}:getInstvar1().
123
The objects that this system provides are pure-functional objects, which is unusual: many object-oriented languages don’t clearly separate the two orthogonal features of late-binding and mutable state. A well-designed language should let you use one without the other, just as Erlang does here: in Erlang, using parameterized modules for method dispatch doesn’t change the way the usual mechanisms for managing mutable state are used. “Instance variables” of parameterized modules are always immutable, and regular state-threading has to be used to get the effects of mutable state.
I’d like to see this feature promoted to first-class, documented, supported status, and I’d also very much like to see it used to structure the standard library. Unfortunately, it’s not yet very well integrated with existing modules like gb_sets, ordsets and sets. For example, here’s what happens when you try it with a simple lists call:
5> lists:append([1, 2], [3, 4]).
[1,2,3,4]
6> {lists, [1, 2]}:append([3, 4]).
[3,4|{lists,[1,2]}]
Not exactly what we were after. (Although it does give brittle insight into the current internals of the rewrites the system performs: a {foo, ...}:bar(zot) call is translated into foo:bar(zot, {foo, ...}) - that is, the this parameter is placed last in the argument lists.)
1 comment May 18th, 2008 tonyg
A touchscreen mod for the Asus Eee 701
My Asus Eee PC has finally received some attention: I installed a touchscreen. The full story is after the jump ..
Continue Reading 4 comments May 15th, 2008 mikeb
Monte Carlo model for Presidential elections
Continue Reading Add comment May 12th, 2008 Paul Crowley
Diff for Javascript, revisited
Last weekend I finally revisited the diff-in-javascript code I’d written a couple of years back, adding (very simple) patch-like and diff3-like functionality.
On the way, not only did I discover Khanna, Kunal and Pierce’s excellent paper “A Formal Investigation of Diff3“, but I found revctrl.org, the revision-control wiki, which I’m just starting to get my teeth into. I’m looking forward to learning more about merge algorithms.
The code I wrote last weekend is available: just download diff.js. The tools included:
Diff.diff_comm- works like a simple Unix comm(1)Diff.diff_patch- works like a simple Unix diff(1)Diff.patch- works like a (very) simple Unix patch(1) (it’s not a patch on Wall’s patch)Diff.diff3_merge- works like a couple of the variations on GNU’s diff3(1)
Read on for some examples showing the library in action.
Continue Reading 2 comments May 9th, 2008 tonyg
E4X: Not as awful as I thought
Long, long ago, I complained about various warts and infelicities in E4X, the ECMAScript extensions for generating and pattern-matching XML documents. It turns out that two of my complaints were not well-founded: sequence-splicing is supported, and programmatic construction of tags is possible.
Firstly (and I’m amazed I didn’t realise this at the time, as I was using it elsewhere), it’s not a problem at all to splice in a sequence of items, in the manner of Scheme’s unquote-splicing; here’s a working solution to the problem I set myself:
function buildItems() {
return <>
<item>Hello</item>
<item>World!</item>
</>;
}
var doc = <mydocument>{buildItems()}</mydocument>;
You can even use real Arrays (which is what I tried and failed to do earlier), by guerilla-patching Array.prototype:
Array.prototype.toXMLList = function () {
var x = <container/>;
for (var i = 0; i < this.length; i++) {
x.appendChild(this[i]);
}
return x.children();
}
function buildItems() {
return [<item>Hello</item>,
<item>World!</item>].toXMLList();
}
var doc = <mydocument>{buildItems()}</mydocument>;
Programmatic construction of tags is done by use of the syntax for plain old unquote, in an unusual position: inside the tag’s angle-brackets:
var tagName = "p";
var doc = <{tagName}>test</{tagName}>;
So in summary, my original expectation that E4X should turn out to be very quasiquote-like wasn’t so far off the mark. It’s enough to get the basics done (ignoring for the minute the problems with namespace prefixes), but it’s still a bit of a bolt-on afterthought; it would have been nice to see it better integrated with the rest of the language.
2 comments May 7th, 2008 tonyg
Ubuntu on EeePC is fairly slick
The instructions were pretty easy to follow (admittedly, after 10 years, you learn where the awkward spots are in linux installations) and the result is a tiny, snappy, fully-working Ubuntu machine, complete with webcam and wifi. The only bit I haven’t got working yet is microphone input to Skype; my bet is that it’s a simple mixer setting.
Update: This page has instructions on how to get the microphone working. The key piece of information is the “Capture Switch” settings.
3 comments May 7th, 2008 tonyg