Archive for June, 2008

George Carlin on work

George Carlin’s passing this week was a suprising loss, not least because I always thought of someone who told hard truths in a way that made you laugh.

One of his famous quotes is:

Most people work just hard enough not to get fired and get paid just enough money not to quit.

In my experience this is true of too many software developers, and it’s a tragedy, particularly because when done well, software is about creative problem solving, which is the antithesis of “working just hard enough.”

“But, I’m Not Dead Yet”

I keep getting reports that TurboGears seems to be stagnant, and from the inside that just does not make sense: We’ve had 202 checkins in the the last 30 days. Which, by way of reference, is a tiny bit more than Django’s 183.

So, that seems to indicate that we’re definitely not quite dead yet. But, even that’s only half the story.

If you pull in checkins to a few components that are external to TG, but internal to Rails and Django (template language, ORM, forms + form validation, etc) that number grows significantly:

  • 58 — SQLAlchemy — ORM
  • 38 — Genshi — Template Language
  • 113 — ToscaWidgets and tw.forms — Widgets, and Widget based Forms

Thats 353. Now if you pull in the changes to Pylons, related middleware and helpers, there’s another big jump:

  • 56 — Pylons
  • 106 — Paste
  • 55— WebHelpers
  • 36 — Routes
  • 6 — Beaker

Now, of course there are other components which are actively contributing to the development of stuff that TurboGears users get as part of the package. But even ignoring all of that we’re looking at over 600 commits in the last month.

TurboGears ecosystem growth:

There’s also a growing ecosystem of turbogears stuff, and it’s all moving forward very quickly too.

Here’s a quick sample of projects I’m watching:

  • 26 — TGTools
  • 33 — tw.openlayers
  • 31 — tw.dynforms
  • 4— dbsprockets
  • 7 — tw.dojo
  • 16 — tw.jquery

There’s a new TG2 based CMS in the works too (http://code.google.com/p/lymon/). And there are a couple more very interesting projects that have not yet gone public.

Splitting our attention:

One thing that’s worth mentioning about all of this is that we’ve intentionally split our efforts over two areas:

  1. Evolutionary improvements to TG 1.x
  2. A new core in TG2.x

And that slows us down a bit. But it’s important because we want to take care of our installed userbase, and to try to grow into new areas at the same time. Doing one or the other would be way easier, but ignoring either one would be a huge mistake.

If it wasn’t for all the amazing stuff going on in the WSGi component world, TG2 would not be possible.

But because the wider python web community is developing new ways to work together around the WSGI model, tg2 development has been moving forward very well. And that’s one of the reasons why I’m so sold on the “component” model of framework development. Sure, it would be nice to have everything under one roof, and to have a stronger guiding hand on the whole process. But it’s not worth giving up all the innovation that happens “at the edges.”

TurboGears 1.x progress

On the evolutionary improvement front we’ve done lots to support SQLAlchemy, Genshi, ToscaWigets, DBSprockets, improve Json support, and created a brand new testing infrastructure, improved out authorization system, and otherwise made lots of positive changes. We’ve also had a half dozen new releases, with feature enhancements, averaging about a release a month.

TurboGears 2 progress

TG2 constitutes our revolutionary front, we’ve added many, many new features, and tools, and have maintained very significant API backwards compatibility, improved performance, and entirely redesigned the core of TurboGears. And we’re approaching our first alpha release in the next few days, so I don’t want to belabor the new stuff here.

What we need to do better:

I understand that not all of this work has been very visible, and that’s our fault, for not engaging the wider python community better, and it’s my fault in particular for not getting the TurboGears 2 work out into the wider world more quickly. Which is something I definitely intend to change in the very near future!

So, if you’re a part of the TG community, and you’re site/project needs to be better known, let me known, let me know. We need to raise the profile of some of the very interesting stuff that’s being done in the community, because outsiders seem to think that we’re not moving very fast, while insiders talk to me about the “blistering pace” of development.

Thinking about Auth*

One common piece of user feedback from the TurboGears 1 community:

Authentication and Authorization are somewhat too closely tied by identity.

At the same time, it was very, very nice to be able to offer people an out of the box solution to their total auth needs, and most brand-new web-application projects use a local database for both kinds of Auth, so TG1′s Identity module was good enough as the 80% solution.

TG2 however, while still aimed at making it easy to get started, and easy to build new web applications, is also aimed at solving some of the more “industrial strength” problems that the current generation of “dynamic” web frameworks has not yet addressed. Old locks are cool! In the case of Auth*, we’re basically talking about using pre-existing auth services. Identity supported this, but it was non-trivial exercise to get everything working.

So, in TG2, we’re partnering with the Repoze project folks to build up a simple, standard interface for authentication service providers. We’ve got a plugin for Repoze which adds a simple database authentication provider, and it works great. But at the same time, we want to make database backed authorizatoin as simple as possible, so we’ve also included some Authorization decorators (with the same API as the ones provided by Identity in TG1) and extended our basic Authorization provider to “decorate” the request with authorization information as well as the basic user authentication information provided by the standard Repoze.who middleware.

The nice thing is that while tg.repoze.who provides both Authentication and Authorization, it’s much easier to separate them if you want. Also I have high hopes that repoze.who becomes a standard authorization provider in the WSGI world, so non TurboGears 2 wsgi apps can (and hopefully will) be designed to work with it out of the box.

There are LDAP and other plugins on the way, and the whole system is still evolving somewhat, but tg.ext.repoze.who is looking like a clean and useful library for both Authentication and Authorization, and it will provide a great platform for TG, Repoze, and hopefully other WSGI application and framework people to work together on the Auth problem.

Florent Aide and Chris McDonough have done all the heavy lifting on this, and I’m very excited about what they’ve done.

Thanks guys!

TG2 Status

Well, it’s official now, I have a new job with Predictix, doing open source TurboGears and Python web dev stuff. Predictix is very much invested in helping the TurboGears community to grow and thrive, and I’m proud to be working with their team. And I’m even more excited about the fact that they want me to do work on TurboGears 2 as part of my “real job.”

One of my main goals when looking for this job was to make sure that whoever I worked for was committed to growing and takeing care of the TG development community. And I couldn’t have asked for anything better. Working for Predictix will help me to polish up the good work that’s already been done to get us to a TG2 beta release, and they already have a lot of fantastic stuff that they would like to open source, which I’m really excited about.

TG2 is moving forward like crazy. In the last three weeks, we’ve had two sprints, both of which had several people working on docs, and on adding the last few features needed for the beta, and cleaning up the show-stopper bugs in our ticket system.

I’m a bit burned out by all the activity, but at the same time I’m very excited about where we are going. I think 2008 is shaping up to be a really busy year for the TG dev team. I see my job in the very short term as creating some stability and consistancy in the midst of the firestorm of new development that’s going on. So, my highest priority right now is getting a stable beta release out the door, and helping us to move forward the docs so that anybody who wants to try out TG2 has a stable base to work on.

My plan will be to do releases about once a month for the rest of the year (or until we have a TG2 final release), because there’s a lot going on, and I want to make that stuff available to people as soon as possible.