Posts filed under 'Programming'
This evening, after fighting bitbake (in the form of the capricious “insane.bbclass” class definition) for a good few hours, I managed to get Erlang version R11B-5 running on my new cellphone.
Running the interactive erlang shell on a cellphone is pretty cool. Erlang’s built-in clustering support works fine: I’ve successfully connected an erlang node on my pc to a node on the phone using the USB ethernet support the phone provides.
The base package compiles down to a bit less than 7MB, which is a bit large. The full suite of libraries are another 22MB or so. It’s certainly possible to fine-tune the packaging process to get a smaller distribution, but for now I’m happy developing against what I have.
Update: I’ve posted my changed build scripts to OpenEmbedded’s bug tracker at bug 3014. Here’s a direct link to the tarball, if anyone would like to try it themselves.
September 16th, 2007
tonyg
This (LTU discussion here) is the most exciting programming language implementation I’ve seen in months. Time to learn more about Forth!
September 14th, 2007
tonyg
Erlang represents strings as lists of (ASCII, or possibly iso8859-1) codepoints. In this regard, it’s weakly typed - there’s no hard distinction between a string, “ABC”, and a list of small integers, [65,66,67]. For example:
Eshell V5.5.4 (abort with ^G)
1> "ABC".
"ABC"
2> [65,66,67].
"ABC"
3>
Erlang also has a binary type, a simple vector of bytes. In the rfc4627/JSON codec I made for Erlang, I chose to use binaries to represent decoded strings, as suggested by Joe Armstrong.
All was well - until I came to implement UTF8 support after Sam Ruby got the ball rolling. Binaries will no longer work as the chosen mapping for JSON strings, since strings may contain arbitrary characters, including those with codepoints greater than 255.
It has always been the case that the ideal representation for a JSON string is an Erlang string, a list of codepoints. Binaries are really a bit of a compromise. But choosing strings-for-strings puts us straight back in a weakly-typed position: it’s possible in JSON to distinguish between “ABC” and [65,66,67], but it’s not possible to make the same distinction in Erlang. We’d need to alter the way JSON arrays are represented to compensate.
Possible solutions:
- Map strings to lists of codepoints. Map arrays to tuples rather than lists. Objects remain {obj,[…]}.
- Pros: Terse syntax for strings and arrays, no worse than the Unicode-ignorant mapping
- Cons: Awkward recursion over arrays, either using a counter and the element/2 BIF, or converting to a real list
- Map strings to binaries containing UTF-8 encoded characters. Keep arrays as lists. Objects remain {obj,[…]}.
- Pros: Keep terse syntax for strings, with the understanding that the binaries concerned must hold UTF8-encoded text. Keeps the interface largely unchanged.
- Cons: Codec needs to perform possibly-redundant Unicode encoding/decoding steps to ensure that the binaries hold UTF8 even if, say, UTF32 were the format to be used on the wire
- Map strings to lists of codepoints. Map arrays to {arr,[…]}, as other JSON codecs do. Objects remain {obj,[…]}.
- Pros: Natural operations on strings, natural operations on arrays (once you strip the outer {arr,…}).
- Cons: Converting terms to JSON-encodable form is a pain, since you need to wrap each array in your term with the explicit marker atom.
All in all, I can’t decide which is the least distasteful option. I think I prefer the middle option, keeping strings mapped to binaries and viewing them as UTF-8 encoded text, but I really need to get some feedback on the issue.
September 13th, 2007
tonyg
I am guest blogging here on behalf of CohesiveFT. We work with the excellent LShift team on our joint venture, RabbitMQ.
I’m here to invite you to a Birds of a Feather session this coming Thursday, August 30th, at 8pm, in central London. It is FREE and will last for 45 minutes starting at 8pm, followed by the traditional breakout discussions over a beer.
Please do take a look at RabbitMQ if you have not yet done so. It’s a commercial open source product, available under the MPL 1.1 and implementing the Advanced Message Queue Protocol. AMQP is a new way to do business messaging (ie: “what goes in, must come out“). What’s really cool is that like HTTP it is a protocol instead of a language specific API. This should make interoperability between platforms much easier and less painful (business readers: “systems integration projects take less time and success can be
predicted more accurately”). For more information, please see my list of links
here.
What is the BOF about - and why come? It’s an informal session about RabbitMQ and AMQP, and how they apply within popular environments such as Spring, Mule, Ruby,
AJAX, and other messaging protocols such as FIX.
“Informal” means we’ll be encouraging a conversation between people interested in any of these things. We want to hear from you, and from each other, rather than pushing slideware at people.
Come if you want to:
You can find out details of the BOF here. Ideally we ask you to register via the web site, but late arrivals are very welcome - if you turn up, we shall get you in. The BOF is offered as part of the popular EJUG series of tech talks and as a
tie-in with the most excellent No Fluff Just Stuff conference.
If you cannot come but want to know more about any of these things then you can email us at info@rabbitmq.com.
Thank-you very much - and we hope to see you on Thursday :-)
Posted by Chris on behalf of Alexis Richardson, CohesiveFT.
August 28th, 2007
chris
I’ve recently written an RMI service which has state - transactions. The service is implemented using Sleepycat Java Edition collections, and the transactions map to sleepycat transactions.
The StoredMap class depends on ambient authority: it determines the current transaction from the thread. The methods will all be invoked in on separate threads, so we need to deal with this somehow. This should be pretty simple - surely there will be a ‘join’ method - to join the current thread to the transaction. Nope. So now what?
I’ll just have to keep a thread running for the transaction, and execute transactional code in that thread.
In Java 5, we get Executors and Futures in java.util.concurrent, which make this very simple to implement. In fact, we can add some sugar - I can generate a proxy class for an interface, which executes its methods inside a single thread.
Anyway, I have, and added it to the LShift Java Library. It even mangles stack traces into something you can read.
June 25th, 2007
david
Our jukebox (mentioned previously) received an update yesterday.
To download the code,
There is a little bit of documentation available, and you can browse the code.
June 21st, 2007
tonyg
Java’s concurrency model provides a sophisticated menu of ways in which to shoot yourself in the foot. Many styles and many variations are available. To give a taste of some of the delicacies on offer, here’s the essence of a problem I found in some code I’d written recently.
Let thread one perform the tasks:
- lock C
- wait for event 1
- unlock C
- lock C
- wait for event 2
- unlock C
Let thread two perform the tasks:
- signal event 1
- lock C
- unlock C
- signal event 2
When we were trying to pin down the issue, we found that the code would run through to the end about 10% of the time. The other runs locked up with thread one blocked at step 5, and thread two at step 2.
Java doesn’t make any promises about fairness: specifically, thread one’s release of the lock before reacquiring the same lock need not cause a thread switch, even when the runtime knows there’s another thread waiting for that lock.
There are solutions: both STM and shared-nothing message passing are models much easier to reason about. Let’s hope future revisions to the Java system raise the level of discourse.
June 18th, 2007
tonyg
In normal, direct-style programming in (mostly-)functional languages such as scheme and ML, folding is an operation that crops up all the time in various guises. Most list-manipulation libraries for such languages include implementations of left-fold and right-fold as standard. But what about the situation when you’re programming in continuation-passing style (CPS), such as when you’re writing (or trying to write) a metacircular evaluator? Library support for continuation-passing folds isn’t nearly as common.
Here’s the direct-style left-fold function:
(define (foldl kons knil xs)
(if (null? xs)
knil
(foldl kons (kons (car xs) knil) (cdr xs))))
and here’s the continuation-passing left-fold function:
(define (foldl-k kons knil xs k)
(if (null? xs)
(k knil)
(kons (car xs) knil (lambda (v) (foldl-k kons v (cdr xs) k)))))
Note that kons takes three arguments here, where in the direct-style version, it takes two.
One benefit of having CPS folds available is that they expose more control over the loop. For instance, using a normal fold, there’s no way to terminate the iteration early, but using a CPS fold, your three-argument kons routine can simply omit invoking its continuation parameter (presumably choosing some other continuation to run instead). This means that operations like (short-circuiting) contains?, any, and every can be written with CPS fold, but not with plain direct-style fold:
(define (contains? predicate val elements)
(foldl-k (lambda (elt acc k)
(if (predicate elt val)
#t ;; note: skips the offered continuation!
(k acc)))
#f
elements
(lambda (v) v)))
June 11th, 2007
tonyg
YAHOO has announced that it will soon be offering unlimited storage to users of its free web-mail product. What does it really mean, and what can we learn from this about the marriage between engineering and marketing.
Continue Reading March 28th, 2007
Tom Berger
Here’s a problem that crops up regularly in Web interfaces: having a dropdown whose available options depend on another dropdown. The canonical example, if you like, is the date selector:
This has the obvious (though not serious) problem that one can select, say, February 31st. We’d like to catch those errors by having the values in the “day” select depend on what is chosen in “month”. It’s easy to do this on the server with a round-trip, of course, but that has the problems that the user interface can be in an inconsistent state before the form is submitted; it would be nice to be able to finesse the interface, using JavaScript, to remove that possibility.
We can do this ad-hoc, of course, but that needs specialised code for each instance (well, for each type of data and dependency); a general solution would be better. We’d also like it to be unobtrusive, by which I mean two closely related things:
- The markup is meaningful
- It works without JavaScript
Bobby van der Sluis gives a pretty good general solution. Briefly, the technique involves keeping a full set of the dependent options and picking the appropriate options from it when necessary.
It relies on encoding the dependency as an HTML class: to me it’s a muddy use of class, but the real problem is that it’s not a very obvious way of making the dependency explicit in the markup. I think we can improve on it.
Using OPTGROUP we can produce straight-forward markup that makes the dependency obvious:
<form>
<fieldset>
<legend>Select date</legend>
<label for="month">Month</label>
<select id="month">
<option value="jan">January</option>
<option value="feb">February</option>
<option value="mar">March</option>
...
</select>
<label for="day">Day</label>
<select id="day">
<optgroup label="January">
<option value="1">01</option>
<option value="1">02</option>
<option value="1">03</option>
...
</optgroup>
<optgroup label="February">
...
</select>
Without JavaScript it looks like this:
Here’s my (allegedly) improved unobtrusive linked select box code (it uses MochiKit to avoid some verbose DOM manipulation — just read $(x) as “select the element with ID ‘x’”, and the rest does what it says):
function linkSelects(parent, child) {
var parent = $(parent);
var child = $(child);
var cloned = child.cloneNode(true);
removeEmptyTextNodes(cloned);
refreshDynamicSelectOptions(parent, child, cloned);
connect(parent, 'onchange', function(event) {
refreshDynamicSelectOptions(parent, child, cloned);
});
}
function refreshDynamicSelectOptions(parent, child, optionholder) {
var alreadySelectedValue = (child.selectedIndex >= 0) && child.options[child.selectedIndex].value;
replaceChildNodes(child);
var selectedLabel = strip(scrapeText(parent.options[parent.selectedIndex]));
for (var i=0; i < optionholder.childNodes.length; i++) {
var opt = optionholder.childNodes[i];
if (opt.tagName.toLowerCase() == "option") {
var newopt = opt.cloneNode(true);
if (newopt.value == alreadySelectedValue) newopt.selected = true;
appendChildNodes(child, newopt);
}
else if (opt.tagName.toLowerCase() == "optgroup" && opt.label==selectedLabel) {
for (var j=0; j < opt.childNodes.length; j++) {
var newopt = opt.childNodes[j].cloneNode(true);
if (newopt.value == alreadySelectedValue) newopt.selected = true;
appendChildNodes(child, newopt);
}
}
}
}
and lastly, here’s the working version:
There remain a couple of weaknesses. It relies on the convention of OPTGROUP labels being the same as OPTION text; it’s reasonable, since there is a semantic link between those two things. Also, it doesn’t always get selection right — easily seen in this example, where you’d expect a choice of ‘01′ to persist when changing months. I think those are tweaks away.
February 21st, 2007
mikeb
Next Posts
Previous Posts
