Archive for October 11th, 2005
…and produce a usable package system?
The absence of a standardised and user-friendly mechanism to package &
distribute libraries, and to locate and install such libraries has
been a major stumbling block in the adoption of non-mainstream
programming languages for a long time. I wrote about it here in the
context of Scheme.
Haskell is actually in a much better position than Scheme for
addressing this, since there are far fewer implementations (and a
single one - GHC - with an overwhelming market share), and far fewer
incompatibilities between implementations. I have high hopes for
Cabal. However, so far it seems to have failed to deliver on its
promises, as evidenced by the following snippet from the installation
instructions for WashNGo:
In some setting, it will be complicated to get everything to work with
the database connection. The reason is that some versions of c2hs
(0.13.6 in particular) do not install properly as a package: the
hs-libraries component of the package description is missing with the
consequence that programs do not link properly. If you can fix they
you fly.
Installing newer versions (0.14.2 or 0.14.3) is not possible out of
the box because they require cabal with version >= 1.0.1. Alas, GHC
6.4.1 ships with 1.0. Installing cabal-1.1.4 does not help either
because…
.../c2hs-0.14.3 > ./Setup.hs build
Preprocessing executables for c2hs-0.14.3...
Building c2hs-0.14.3...
Chasing modules from: c2hs/toplevel/Main.hs
Could not find module `CForeign':
Oh well.
That’s exactly the kind of thing that will turn off all but the most
determined geeks from exploring Haskell.
October 11th, 2005
matthias
The SSAX XML parsing- and processing-library provides robust, high-quality XML reading tools for Scheme, but doesn’t include a general purpose XML writer. Over the past couple of years, a few of my projects have had a need to convert SXML-like data to an XML 1.0 external representation, and so I’ve written a portable SXML-to-XML printing library (both a snapshot and a darcs repository). The library has been used with Chicken, MzScheme, and SISC, and currently includes module wrappers for Mzscheme and SISC (or other psyntax-based Schemes).
The library is parameterized over a choice of double- or single-quotes for attribute printing, and can, if required, be instructed to use explicit close-tags when an empty-tag is encountered. It provides procedures for producing a string representation of an XML fragment, for printing an XML fragment directly to a port, and for pretty-printing an XML fragment with indentation. For example,
(pretty-print-xxexpr
(let ((title "My Page"))
(list
'(*pi* xml (version "1.0"))
`(html (head (title ,title))
(body (h1 ,title)
(p "Hello, world!"))))))
produces the following output (don’t mind the invalid XML PI, that’s something WordPress is doing - the actual output from the library is well-formed!):
< ?xml version='1.0'?>
<html>
<head>
<title>My Page</title>
</head>
<body>
<h1>My Page</h1>
<p>Hello, world!</p>
</body>
</html>
You can either browse the code, or retrieve it using
$ darcs get http://www.lshift.net/~tonyg/xxexpr/
October 11th, 2005
tonyg
Background
The data language JSON is a great replacement for XML for many applications. It’s very similar in spirit to Lisp and Scheme S-expressions, as well as to XML: it is a pure data language, with no intrinsic semantics.
XML doesn’t allow direct literal representation of any data types other than strings and XML nodes (to oversimplify slightly); S-expressions don’t allow direct literal representation of anything but atoms, pairs and vectors; and JSON doesn’t allow direct literal representation of anything but atoms, lists, and dictionaries.
There are currently no extensions to XML to remedy this, to allow programmers to extend the XML reader to support literal notation for other classes of object. For S-expressions, however, there is SRFI 10: Sharp-Comma External Form, which allows the S-expression reader to be extended to allow read-time construction of literal objects.
Reader/Writer Extensibility for JSON
I’ve implemented a SRFI-10-like construction for JSON, allowing extension of the JSON reader and writer, which makes it easier to use JSON for things like serialization or high-level messaging protocols.
You can download the modified json.js from here.
The basic extension is the addition of @constructor ... notation to the reader: whenever the JSON parser sees an “@” sign, it parses a single identifier, and then a complete JSON object called the argument. The identifier is looked up in a table of named constructor functions, and if a match is found, the corresponding constructor function is called with the argument. If no matching constructor function is found, a parse error is signalled.
The corresponding extension to the writer is the toJsonString method. If an object that the JSON.stringify function encounters possesses a toJsonString implementation, the method is called and its result is returned as the stringification of the object.
Example
As an example, imagine an application where a client and server shared some notion of a database of objects, indexable by some unique identifier. The client side uses proxies to manipulate these objects, and the proxies send JSON RPC requests to the server (perhaps using XmlHttpRequest) to query and update the database held by the server. There are two ways we could represent one of these database objects: as just its unique identifier, which requires a manual lookup on both the server and client side whenever a reference to the object is to be sent over the wire, or using the @constructor extension.
MyDatabaseObject.prototype.toJsonString = function () {
return "@dbobject " + this.id;
}
An example message from the client to the server might be
{
"action": "retrieveNextObject",
"arg": @dbobject 42
}
When the server parses the message out of the HTTP request (or other transport), it calls JSON.parse with an extra argument:
var requestFromClient = JSON.parse(requestText, {
"dbobject": function (arg) {
return DB.lookupById(arg);
}
});
When the parser sees the “@dbobject ...” part of the input string, it will call the constructor given to JSON.parse, which looks up and returns the object in the database by its identifier, making for a convenient, hassle-free deserialization of a complex object.
October 11th, 2005
tonyg
As part of my recent experiments with Javascript, I happened to want a sleep function. Interestingly, there isn’t one available in browser-side Javascript environments - instead, there are four functions
setInterval
clearInterval
setTimeout
clearTimeout
and they take functions to call after a certain number of milliseconds have gone by. This leads to programming in continuation-passing style (see here, here, here, here, and here):
// ... some code ...
// At this point, we want to pause for one second before continuing.
setTimeout(function () {
// ... code here continues running after 1000 milliseconds have gone by.
}, 1000);
It seems that a (laudable!) design decision has been made for avoiding blocking operations in browser scripting languages. It’s not just sleep-a-likes, either: the decision affects the design of XmlHttpRequest, which is used in AJAX-like server callbacks. The way you receive the server’s response to an XmlHttpRequest message is by supplying a callback function. This again leads to a continuation-passing style of programming - see the definition of AJAJ.proxy in this file. Here’s an example of AJAJ.proxy in action, as part of a simple AJAX+JSON chat application:
function rpc(action, arg, k, kfail) {
var urlPrefix = (new RegExp("(.*/)([^/]+)")).exec(document.location)[1];
AJAJ.proxy(urlPrefix + "chat-server/simple-chat.asp")
({action: action, arg: arg}, k, kfail);
}
function consoleRpc(action, arg) {
rpc(action, arg,
function (r) { console.recordOutput(formatConsoleRpc(action, arg),
JSON.stringify(r)); },
function (e) { console.recordError(formatConsoleRpc(action, arg),
JSON.stringify(e)); });
}
The two functions k and kfail passed to the rpc function serve as the success and failure continuations, respectively. If the XmlHttpRequest succeeds, then the function starting with “function (r) { ...” will be called with the response from the server; otherwise, the function starting with “function (e) { ...” will be called with an exception object that reports on the failure. We’re using closures to extend the available control structures of the language.
October 11th, 2005
tonyg
The code in this package (not only a snapshot, but also a darcs repository) was originally developed for
two reasons: for use in teaching a course in Javascript
as part of a guest lecture series for the University of
Westminster, and as a consequence of my personal
interest in experimenting with modern client-server
programming techniques.
It’s a very, very simple chat application using XmlHttpRequest to communicate in an AJAX style with an ASP 3.0 server that processes JSON-RPC-like requests and manages a simple shared database.
The reason ASP 3.0 was chosen is because it was the
server-side environment in place at the University
laboratories for the course I was teaching. Programming
in JavaScript on both the client- and server-sides
simultaneously was a pleasant experience; modern
environments such as Helma have a
lot going for them.
If you take a look at the code, you’ll see that I use the term “AJAJ” often: the idea is to take the core elements of AJAX, but replace the unwieldy XML with something lighter-weight and better fitted to Javascript programming. In this case, the “X” for XML is replaced by “J” for JSON, which is a lightweight data language that is particularly convenient for working
with Javascript.
Code overview
The application depends on a few third-party libraries that make programming in Javascript a little more pleasant. These are the parts of the application that I wrote:
simple-chat-client.html: Contains the HTML
user-interface to the service, and the basic
client-side Javascript code for driving the specifics
of the chat service.
chat-server/simple-chat.asp: Contains the server-side
code responsible for responding to the client’s RPC
requests for chat-server functionality. Does not
itself present any HTML at all: it only speaks AJAJ,
accepting JSON requests and replying with JSON responses.
js/ajaj.js: Implements the client side of the AJAJ
RPC library, building upon json.js.
chat-server/ajaj.asp: Implements the server side of
the AJAJ RPC library, building also upon json.js (in
the form of json.asp). The only part of this file not
authored by LShift is the contained VBScript function
BinaryToString, which was written by Antonin Foller
and comes from http://www.pstruh.cz.
js/console.js: A reusable Javascript DHTML evaluator.
Useful for debugging and interactive development of
Javascript-bearing HTML pages.
js/util.js: A collection of utilities lending, among
other things, a functional-programming flavour to Javascript.
chat-server/jsondb.asp: A simple (toy) transactional
database driver for ASP 3.0 using the filesystem as a
data store.
The following files are mostly library code from third parties:
October 11th, 2005
tonyg
Over at eighty-twenty I recount a couple of recent random excursions into various bits and pieces of code.
Besides those developments, I also spent some time on Sunday morning modifying Dorai Sitaram’s pregexp version 20050502 to operate
over streams as well as strings, so that I could use it for lexing
arbitrary character sources (for instance, with the packrat parser library I’ve been developing).
The basic interface, after the patch, is now either the standard
(pregexp-match <pattern> <string>)
or the new
(pregexp-match <pattern> <stream>)
where streams are created with
(pregexp-make-stream <generatorfunction> <seed>)
I’ve also added a procedure pregexp-match-head, which is like
pregexp-match-positions except it only matches at the very beginning of
the input string or stream; pregexp-match-head behaves like Python’s
re.match, where pregexp-match-positions behaves like Python’s re.search.
I haven’t modified pregexp-split, pregexp-replace, or pregexp-replace*,
partly because I have no need for them for my application and partly
because I’m not sure what their behaviour should be: should
pregexp-replace, when given a stream, answer a stream, or a string? In
the case of pregexp-split, since it has to examine the entire input in
any case, supporting streams seems unnecessary. (Perhaps I should have
included a pregexp-stream->string utility, though.)
The patch against version 20050502 is downloadable here.
Thanks to Dorai for a great library!
Update: I noticed a bug in the first revision of the patch. I’ve updated the links in the article above to point to the new patch.
October 11th, 2005
tonyg