technology from back to front

Web Development with Python

For a new web project we’re working on, we wanted to use a dynamic environment. Initially we considered Ruby on Rails, but after using it for some time decided the shortcomings of the framework and the language outweigh the benefits. We’ve resolved to use Python, a language we feel very comfortable with, and I went to test several pythonic web components, in particular the stuff that gets bundled with TurboGears and web.py (we briefly considered Django, the web framework adopted by Python’s creator Guido van Rossum, but identified all of the same problems we had with Rails, so we didn’t spend too much time with it).

TurboGears

TurboGears is a collection of Python web components, rather than a top-down complete framework like RoR. It uses CherryPy for the controller (using WSGI for low-level infrastructure), SQLObject for database access and manipulation, [old link: http://kid-templating.org/ ] Kid for templating and [old link: http://mochikit.com/ ] MochiKit as its main Javascript library. On top of that TurboGears provides several integration tools which are supposed to turn this collection into a ‘framework’ (a skeleton generator, some simple web-bsed administration, a shell preloaded with the project modules, etc).

My experience working with TurboGears has been much better than working with RoR. Partly, that’s because I know and like Python better than Ruby, but mostly the approach is saner and much less demanding – there’s very little magic, and it’s quite easy to understand how to go about implementing your ideas. With TurboGears I was able to complete the same amount of work I did in my RoR test in ~20% of the time!

I wasn’t very impressed with the framework / integration parts of TurboGears. The quickstart tool (for building the skeleton) did some stuff that was quite obvious and I could have completed myself with little effort, and some stuff that I didn’t expect or want it to do (just like RoR). The shell is convenient, but again, I felt that I paid a price in flexibility without gaining much – after all that’s just the Python REPL with some modules imported. The build and deployment tools are a great idea, but they don’t seem to be powerful enough, and extending them may or may not be easier than just doing it ourselves using a build tool like SCons or Make, or simply writing straight python scripts.

Components

Data Access: SQLObject – When we first tried out SQLObject we found it enticing – the interface is simple and easy to understand and the features it provides looked like a perfect match for our needs. After working with SQLObject for a while, however, we were shocked to discover that it’s hardly adequate. SQLObject’s model of working with databases is naive, at the very least – for every attribute you set (on a data-bound object) the mapping issues a separate SQL update statement and what’s worse, if you plan to use SQLObject’s cache (which is the only way to use it efficiently to query the database) you can simply forget about transactions. I find it hard to understand how TurboGears can get away with that – after all, with optimization and ACIDity thrown out of the window, it’s hard to see in what way saving objects to a relational database is any better than simply pickling them.

Templating: Kid – Kid is an attribute language (like ZPT) – it allows you to generate XML by inserting special attributes instructing the processor how to do dynamic things (like looping, inserting values and so on). Unlike ZPT, Kid is much more pythonic – almost all constructs map directly to control flow structures known from Python. I was able to start working with Kid without having to read any of the documentation and got good results immediately. (Kid is now slowly being replaced by Genshi as the state-of-the-art in Python XML templating, but for our needs we felt that Kid is just good enough).

Javascript / AJAX: MochiKit – I had quite a lot of previous experience with MochiKit, so I didn’t bother testing it in this context. MochiKit shines in comparison to other JS libraries (Prototype, Dojo, YUI) – it is very high quality, got excellent documentation, and anyone with experience using Python or any functional programming languages will feel very comfortable using it. I prefer it to any other JS library.

Controller: CherryPy – I didn’t like CherryPy, although it did look adequate. Particularly, CherryPy relies on mapping Python objects and attributes to HTTP access more than I would like it to. My experience working with web.py was better (the library is richer and provides some neat facilities for rapid development, the mapping of requests to handlers is saner).

We ended up deciding to mix and match – we use web.py as a controller. Instead of an ORM we design our own model objects and connect them to the database using web.py’s data access facilities. We use Kid for templating and some other Python libraries for supporting more specific features of the website.

Conclusions

  • Dynamic languages and frameworks are the hot thing in web development for a good reason, but RoR’s lead is not wholly justified. 1
  • Python has a huge collection of frameworks and components for web development, but the quality varies quite a lot. Choosing the right mix of tools and libraries can be difficult, but being able to mix and match gives you great power and flexibility.
  • TurboGears is a nice take on this idea, but the choices made are dissatisfactory.
  • Web.py is an excellent start, but to do useful things you’ll have to augment it with bits from other sources.

[1] I still owe you a detailed critique of RoR – possibly in a future post. In the meantime, don’t take my word for it – lot’s of people swear by it – you too may find it to be the right tool for your project.

by
Tom Berger
on
15/11/06
  1. Jacob Lee
    on 15/11/06 at 8:49 pm

    You should perhaps look into SQLAlchemy. I’ve also found SQLObject to be far too limited in terms of the subset of relational operations that you can perform with it — I had to fall back on raw queries and the Python DB-API way too many times. I haven’t used SQLAlchemy yet, but I’ve heard a lot of good things about it, and it seems to handle just about everything one would want to do with a database.

    (The rest of the choices are something of a personal preference. I found Kid a little painful in comparison to Django; for controller mapping, I like the Routes-esque design of Django and web.py, but can live with Turbogears’ method.)

    I look forward to reading your opinions on RoR.

  2. … SQLAlchemy … seems to handle just about everything one would want to do with a database.

    And that’s exactly why we’re not using it. We did try SQLAlchemy as an alternative (more specifically, it’s ActiveMapper extension) – SQLAlchemy looks like a much more serious project, but it’s huge and we just didn’t feel like we can afford to get into it without knowing more about it (for writing production code).

    Also, there is already a whole class of products that does everything one would want to do with a database, namely, DBRMS – if you want me to replace them with something new, it better offer me some tangible benefits – writing queries in Python instead of SQL is not necessarily a benefit.

  3. what about ZOPE, i think it’s offering a lot to consider .

  4. Yeah I agree that Rails isn’t for everything. That doesn’t mean that it’s bad it’s just not made to fit every case and it’s actually not meant to be a “one size fits all” that as DHH has explained many times.
    I get Python. I get what it’s trying to do and I support it. But I have tried using it and I have found that I just don’t find it comfortable. I really didn’t like the whitespace and a few more things on how it forces me to do things like Python wants it done. But thats just my feeling. So although I checked out Turbogears and more I just always stopped on the Python part.
    Perl’s Catalyst framework felt fresh in some respects but it’s installation and it’s 1000 packages and problems meant that installing it could take and hour or more compared to RoR’s 5 minute installation time.
    So I’m sticking with Ruby for now. Often I use RoR. For smaller utilities type things I use Camping (Think of it as cgi-bin like framework. Each tool should fit in a single file) and I have taken a good look at the Nitro Project.
    In my opinion then Ruby’s biggest problem is Apache’s mod_ruby which is utterly unusable. The favourite solution of many is to use other web servers that are faster anyway but Apache is big for a reason as it supports doing just about everything.

  5. Zope is in a class of it’s own. But it’s not the best for a public web applivation though. It’s got an amazing database freamwork and it’s strange in many ways. That’s why it’s used in many powerful databases such as the inventory for the Navy. But it’s VERY easy to get into trouble using it. You only use Zope when you know exactly what you are doing and when you know Zope very well. You don’t just look at a project and decide to use that Zope thing for it.

  6. You should give Pylons a roll. It’s very similar to TurboGears, but it gives you better control and isn’t CherryPy based.

  7. FYI, CherryPy 3 includes a Routes controller, and if you don’t like that or the builtin, you can roll your own. See http://www.cherrypy.org/browser/trunk/cherrypy/test/test_routes.py for example code.

  8. Look forward to reading the Rails critique.

  9. Bob, if Pylons is as good as MochiKit, then I already like it :)

  10. Hubert Plociniczak
    on 10/01/07 at 3:12 am

    Pylons main problem is its dependency on third-party plugins, libraries which themselves are quite new. Even because of that it seems to develop rapidly. I really liked the idea of good support for different template engines or SQL databases.
    Besides because it is built around WSGI it is faster to implement small changes/adjust it to the project rather then getting into the code of other frameworks.

  11. I agree with Robert – check out cherrypy 3 with its routing mechanism. I find that cherrypy does the best job of staying out of my way of all the web frameworks I’ve tried.

    It’s small and simple enough, too, that you can read its code when you need to figure out how something works.

  12. Out of curiosity, what didn’t you like about Django? We’re always looking for constructive criticism…

  13. teki321
    on 25/03/07 at 2:25 am

    I just read this: http://jesusphreak.infogami.com/blog/vrp1

    Update by Paul Crowley: this comment isn’t blogspam – it links to an article entitled “Python web development and frameworks in 2007″.

  14. Jaramy on better web business solutions
    on 25/03/07 at 1:58 am

    James Bennett,
    sometimes people needs “lesser” framework than “bigger” framework – esp when making very specific app

  15. Sometimes I am wondering why people are so black or white. I love/hate Python vs Ruby.

    It is perfectly legitimate to prefer Python over Ruby but it is something else to say one is better than the other …

    Anyway both seems kind of closed. It is the little “the shortcoming of the framework and the language” that bothers me the most. Mixing in the same sentence two completely different ‘shortcomings’ is not a good idea; at least when you give some respect to programmer languages in general.

  16. @Pit,

    Are you suggesting that there is no way in which one programming language can be better than another?!

  17. Daniel Kwiecinski
    on 30/05/07 at 6:06 pm

    Are you still using Python for building web apps? Have you considered any functional language for this purpose?

    As I heard LShift advocates usage of Erlang, the framework worth to consider could be Erlyweb. It is MVC component based framework with some RoR-like magic, build on top of Yaws. Yaws supports dozen or even hundreds of thousands of concurrent connections and it benefits of Erlang’s features.

  18. @Daniel,

    That’s a very good question!

    We have used functional programming languages before for web development (SISC Scheme for NMK, Eralng for TonyG’s jukebox) and I would have loved to use a functional language to do development for other sites too (LISP / Scheme especially seem like excellent languages for web stuff). A few things to keep in mind, though:

    1. Python is, to some extent, a functional language. It isn’t pure, and anonymous functions are too weak to my taste, but if you like the functional paradigm Python allows you to follow this path. I find that most of my Python code, especially for the web, is mostly functional.

    2. There’s more to a programming language than just the language itself. When developing applications you often need to rely on existing code and tradition. Python has a fantastic community and a huge collection of libraries, which no other (dynamic) language can compete with right now – certainly none of the purely functional languages available. If we were to choose a different, functional language, it is likely that any gain we’d make in productivity would be offset by losses due to having to write a lot of code from scratch that for Python is readily available in the form of libraries, frameworks, free software and documentation.

    So, Python was a great choice for a few web applications we’ve built, and I’m definitely going to continue with it as my primary choice for now, but I’d definitely watch the functional arena closely – the way things are developing I wouldn’t be surprised if we’ll have an adequate competitor soon (Haskell? Erlang? R6RS? Arc?)

 
 


5 + six =

2000-14 LShift Ltd, 1st Floor, Hoxton Point, 6 Rufus Street, London, N1 6PE, UK+44 (0)20 7729 7060   Contact us