Recently, as part of RabbitMQ server development, we ran into an interesting issue regarding Erlang’s per-process garbage collection. If a process is idle — not doing any work at all, simply waiting for an external event — then its garbage-collector will not run until it starts working again. The solution is to hibernate idle processes, which causes a very aggressive garbage-collection run and puts the process into a suspended state, from which it will wake when a message next arrives.
We were implementing a form of flow control in RabbitMQ server, essentially equivalent to XON/XOFF, and when a channel was paused, we found that its memory usage wasn’t decreasing in proportion to its decreased activity. As soon as we started hibernating processes, the memory usage dropped right back.
Over the last little while, I’ve started to suffer from lack of space on the hard disk in my laptop, which is ridiculous, since there’s an 80GB disk in there and there is no way I have that much data I need to hang on to. I decided to do something about it last week. The main part of the problem was to figure out what was eating all the space:
du tells you exactly what’s using how much, but it’s hard to get a feel for where your space has gone by scanning through pages of
du output. So I built a program to help.
spaceviz is a small Python program that takes the output of
du -ak, and builds you a picture and HTML client-side imagemap of your space usage
Running it against the output of
du -ak / showed me very clearly where all the space had gone: not only did I have a few seasons of various TV shows on my disk (which I already knew were there), but I had 11 GB of unneeded gzipped RDF data left over from a project that finished earlier this year (that I had forgotten about). Instant win!
To run it for yourself, check out the mercurial repository
http://hg.opensource.lshift.net/spaceviz, and run
make veryclean all ROOT=/
ROOT=/ with a definition of
ROOT that points at the directory tree you want to generate usage data for. The makefile will take care of running
spaceviz.py for you. Edit the settings for
spaceviz.py to change the dimensions of the generated picture.
The program runs not only on Linux without fuss, but also on OS X so long as you have the
netpbm port installed to convert the python-generated PPM file to a browser-accessible (and much more efficiently compressed!) PNG.
I spent a few days in Wellington recently with my 12-year-old brother. We somehow got to talking about ray tracing, and so we ran through a bit of linear algebra (vectors, normals, and dot products, basically) and built ourselves a raytracer in Python as a lark. We collaborated on the Vector and Point classes, and I filled in the rest of the program. There’s still a bug in the checkerboard code (see the odd horizontal lines in the reflection of the checkerboard in the yellow sphere?) but it’s not too awful for an afternoon’s idle hacking.
The code is contained within raytrace.py, and here’s the output it generates:
Wouldn’t it be nice to have classes like Vector and Point as part of the standard distribution of Python? Even better, though, and perhaps less subject to bike-shed debates, would be the inclusion of a simple, small PNG-writing module like this one as part of standalone base Python!
You are currently browsing the LShift Ltd. blog archives for October, 2008.