DjangoCon (Part 2)

As I mentioned in my last post, I did a talk at DjangoCon a couple weeks ago, which has proved to be a little bit controversial in some circles (though the biggest critics are people who admit they didn’t actually watch it).

To see the part of the talk I’m talking about in this post jump forward to about minute 19 of the talk.

My previous post talks about almost all of the stuff that I consider controversial, and is basically an argument from history that very-large monolithic code-bases can get into trouble, and cause community fragmentation by separating out those who use “monolith x” from those who don’t. This is particularly true when the monolith is not particularly modular internally. This was the case for Zope 2 and Django seems to be following them down that path to some extent.

But today, I’d like to shift focus from the past to the present, and from social lesions to a famous bumper sticker I’m pretty sure most of us have seen:

SHIT happens

But since I don’t want to be too negative, let’s rephrase that:

GOOD shit happens

And I’m actually not interested in how great the django core-dev team’s drinking abilities are, or how wonderful their social lives are.

I’m interested in the good shit that’s relevent to making developing web applications better, so perhaps we could use this bumper sticker:

Innovation Happens

Django’s dev team has done some great stuff, that has shown that there’s lots more that can be done to make web developmet easier for python programmers. But at the same time not everybody uses Django and lots of problems that Django is currently trying to solve have already been solved by other people.

This argument isn’t really mine, it’s an argument made in the book Innovation Happens Elsewhere by Ron Goldman & Richard P. Gabriel. In the book they argue for using open source software so that you can harness the innovation that’s happening elsewere.

But even for Django, which is open source there’s no cornering the market on innovation, so we could say:

Innovation Happens Everywhere

It’s happening in Django, and outside of Django. It’s happening wherever there are smart people who want to make things better.

Here’s a quote form Goldman and Gabrial’s book:

Silly as it sounds, this is the brutal truth:

Regardless of how smart, creative, and innovative
you believe your organization is,
there are more smart, creative, and innovative people
outside your organization than inside.”

Innovation Happens Elsewhere

But, Django has a policy that they do not depend on anything else. This means that they have to reproduce all the innovation that happens elsewhere in the python web community in order to keep up. Perhaps this is OK, I mean Django is big, and they are innovating themselves, at least that’s the argument that some are making.

I think it’s a bit short-sided though, there’s a lot going on in TurboGears, repoze, SQLAlchemy, Pylons, Zope, etc. And all of those people are starting to work together more and more. They are recognizing that WSGI and reusable libraries makes python web development more flexible than ever before. If anybody thinks web development 10 years from now will look much like it does today has their head in the sand.

In the talk I just mentioned a few limitations in Django that people mentioned in Saturday’s talks. And I took one smart person outside the django community, Mike Bayer, and showed what he’d already done to solve those problems.

  • Beaker: Mighty+Beaker solved the dog-piling problem with caching in django’s cache layer
  • Beaker: encrypted cookie sessions (actually this one was done by Ben Bangert)
  • SQLAlchemy: Batched commits (the unit of work pattern)
  • SQLAlchemy: Multiple database support
  • SQLAlchemy: Horizontal partioning (AKA, sharding)

Of course there are lots of others, who are also doing lots of innovative things that solve problems Django users have right now. And there’s no technical reason Django and the Django developers can’t use that stuff.

What this means for Django

My fundamental argument in this section of my talk is that Django users should start taking advantage of things that the rest of the python development community has been building. Just because Django has a no dependency policy doesn’t mean you should. And it would help the whole community to grow if Django components were easily decoupled from the framework as a whole.

I would love it if the Django ORM could be used outside of Django in an easier way — it’s a very powerful and flexible “Active Record” style ORM — I’d even go so far as to say that with it’s generative query syntax, and the recent queryset refactoring work it’s the best and most thoroughly documented Active Record type ORM in python today. But installing it and using it requires installing Django, and dancing an interesting (though not particularly complicated) configuration dance.

More on the third part of my talk which focuses on the future of Django coming later this week.

4 Responses to “DjangoCon (Part 2)”

  1. I think you’re right on many counts.

    However, I think you’re being mislead to think that Django is talking about the framework dependencies when it uses the slogan, “tightly integrated, loosely coupled.”

    What it really means when talking about Django is that the framework is tightly integrated, but the application code written on it is loosely coupled.

    For example, I write a bunch of code for a forum built on Django. That code lives on its own as a python package. It has only a handful of imports to do from Django in order to work. I can then pack and re-use or even distribute my new forum package. Anyone running Django can import the package into their settings file and it will just work. They can then specify their own template path and start developing their own forum templates without touching my package’s source.

    That, I think, is a very nice feature.

    Contrasted with Turbogears 2 which I just set up the other day — I’m starting with a huge file-system structure and a tonne of boilerplate code. I just entered the ‘paster quickstart’ command and already have more code than most Django apps I’ve ever had to write. There’s a lot of code I would assume should just be in the tg package, but for some reason it’s polluting my application source. What’s with all the boilerplate? Why can’t those controller base classes live in tg.controllers?

    Boilerplate makes babies cry.

    However, Django could definitely be better off if it would open up to outside developments like SQLAlchemy (which is being worked on AFAIK) and the various other template languages and dispatch systems (though I think Django’s dispatcher is the best thing since sliced bread… really not a fan of the active routing method).

    If you are getting lots of flaming it might simply be a mis-communication issue.

    Good talk though overall — I did sit through all 50mins of it on youtube. ;)

  2. Well, I agree that having reusable app components build in Django is a very useful thing, and making that easy is that’s one of the goals of TG2 in the next iteration.

    There’s obviously a balance between providing easy ways to customize things in the quick started template and keeping the code generation under control. TG2 started with the same template as Pylons, and we’ve been slowly but surely decreasing the amount of code in the quickstart template and moving somethings into TG2.

    I do think you may be confused about how the base controller works, it’s just there as a stub for you to use to add customizations too — the controller in lib.base is very, very small and just subclasses a controller in TG2. It’s purpose is just to make it obvious where such customizations should live.

    As I mentioned, most of the flaming is coming from people who haven’t actually watched the talk and are making interesting assumptions about what I must have said, or what motives I must have had.

    I’m not worried about it though, it’s just part of life, and really it hasn’t been that bad.

  3. 3mike bayer

    wrt the paster setup process, all the code generated by paster setup is not “boilerplate” since you didn’t have to create it yourself. It’s all there for you to modify. Once your app gets going and you need to customize its overall functionality, having the whole middleware configuration and everything else transparently visible and outside of the installed libraries is gold. It’s not at all a new idea and is highly analagous to a framework like Struts2. Granted, it’s not ever going to be big with the PHP crowd but as the J2EE community begins to give Python a second look, the Pylons/TG approach will appeal to a broad segment of that base (as it is here, in a shop where we have Pylons and Django apps, the Java devs prefer the Pylons way for its transparency and configurability).

  4. @mark:

    Fan boys will be as they are. It’s good on you to not get involved. ;)

    I think I did misunderstand the purpose of lib/ at first — but that’s just one reason why I don’t like boilerplate code. I didn’t write it and if I am to understand what’s going on in my application I have to know what it does.

    To be fair, it doesn’t look like the TG2 boilerplate does a whole lot on its own, but there’s just so much of it that it’s a little disorienting for new developers despite their level of expertise.

    Anyway — I’m looking forward to giving TG2 a go on a small project.

    Good work again on the talk.


Comments are currently closed.