Archive for June, 2007

TurboGears 1.1 and Beyond

I just posted this to the TurboGears announce list. We’ve been working hard to come up with the right plan for how to move TurboGears forward as quickly as possible, while giving people the features they need, and I think we’ve got a good plan.

Recently there have been a number of requests for more clarity about the future of TurboGears, and since I’ve been involved heavily with a couple of experimental projects designed to help us explore our options, I’d like to help clarify things as best I can.

A bit over a week ago several of us decided to get together in Atlanta this past weekend and hack on an experimental Pylons/TurboGears integration project. We wanted to discover if we could re-implement the TurboGears API on top of paste+pylons in a way that would make TurboGears better.

The goal was not just to re-implement things, but to see if that re-implementation actually improved the readability, flexibility, and ease-of-mantinance of the TurboGears project. In other words, we wanted to make TurboGears: easier for new developers to work on, easier for us to maintain, and more flexible so we can take advantages new python web developments in the future.

On all counts, I think we can now say that this experiment has been a huge success. And after lots of good discussion with Alberto, Kevin, and Ben Bangert from the Pylons project, the way forward is now much clearer. The results of the sprint will become the basis for TurboGears 2.0.

To make life easy for people who want to install TurboGears 2.0 alongside the current version we will be creating a new package called tg for TurboGears 2.0. And Alberto is planing to promote it out of the pygears branch into trunk today.

The new tg package will implement a very large percentage of the current TurboGears API, and thus is intended to provide an very easy upgrade path for current TurboGears users. In particular current controller code should be very simple to port, and Kid, Genshi, SQLAlchemy, and SQLObject code will be supported, so most of your template and model code will require almost no changes, or will be be usable as is.

There’s still work being don on TurboGears 2, and it’s hard to predict a release schedule in open source projects. But if you pressed me I would say that I expect to see a beta in the next couple of months. It could be earlier, it could be later… If you’re the adventurous type and don’t want to wait until then can check it out of subversion today, but expect a lot of rapid changed.

At the same time the turbogears development team will continue to maintain and support TurboGears 1.x for our current users. We want to accommodate users who create new TG 1.0 projects for a long time, but we also want to make it easy for those who want the latest and greatest stuff to get started with TurboGears 2.0 as soon as possible.

There will be a TurboGears 1.1 beta release in the next few weeks, which will fully support the current API, and will switch the default templating engine from Kid to Genshi, and the default ORM from SQLObject to SQLAlchemy. This release will not include an upgrade to CherryPy 3, because that has required backwards incompatible changes, and has taken much more work than expected. If someone wants to take up the standard, there may be a TurboGears 1.2 wiith CherryPy 3 support, but as far as I know nobody had volenterred to do this.

And of course, SQLObject and Kid will remain supported in 1.1, to give us full backwards compatability with current applications. Catwalk and Model Designer will remain SQLObject only applications in 1.1.

And that brings us back to TurboGears 2.0. In addition to the current TurboGears API, the new tg package will support all kinds of new features that will make our user’s life easier. Many of these features are the direct result of our integration with Pylons, because we’ll be getting access to a lot of great stuff they’ve already done. And we’ll be sharing almost all of our infrastructure code with another group of great developers.

The end result is that TurboGears users that have sites that need to scale up, will have have access to world class caching options (including memcached support), as well as a number of other performance enhancing features and options. For the majority of us who are more interested in rapid development, and deployment flexibility, than raw performance, TurboGears 2 will include all kinds of new features. TurboGears 2 will also implement a more flexible Object Dispatch system, as well as direct Routes integration, so it will be even easier to get exactly the URL’s you want.

There’s a lot more going on too like 1.1 TurboGears 2 will make Genshi and SQLAlchemy the defaults because they provide a better experience for the majority of web developers. We won’t be forcing our opinions on anybody, and other ORM’s and templating engines will be supported, but we’ll be focusing all our documentation effort on those defaults. We want to make very sure that our documentation tells a clear and compelling story about how this stack of components makes web development easier.

At the same time, we’ll be working to support people who want to develop and maintian sub-projects for automatic CRUD (like Catwalk or the Django interface) outside the TurboGears 2 core. Our hope is that these tools will be broadly usable by anybody who is working withing the context of a framework that implements the Web Server Gateware Interface standard. Ultimately I think Pylons, TurboGears, Paste, Zope, and all kinds of other python web developers toolkits will be working together on creating great tools for rapid development.

Thanks to Pylons, Paste, SQLAlchemy, Genshi, Babel, and a whole host of other contributions from around the web TurboGears development is getting easier and more fun. I for one, am really looking forward to all the good things that are in the pipeline!

I’m looking forward to some spirited discussion of the TurboGears 2.0 plan, and the plan for TurboGears 1.1 on the TurboGears trunk mailng list.

(http://groups.google.com/group/turbogears-trunk)

Experimentation and Sprints in TurboGears

A bit less than two weeks ago Jonathan LaCour and I were driving down 400 into Atlanta for the Atlanta Python Users group meeting and we were taking about the pace of development in TurboGears recently.

And even though there is evidence that there’s been reasonable activity on the project recently. We both had general sort of feeling that the project was stuck and not moving forward the way we’d like. As a whole the project was focused on trying to provide maximum backwards compatibility as we created TurboGears 1.1, but that had turned out to be lots and lots of relatively uninteresting work, and given our limited resources was just not progressing that quickly in terms of new featurs or new releases.

As we talked about the core TurboGears we used day in and day out at work it became more and more obvious we could probably implement that same core API in top of Paste/Pylons relatively quickly, perhaps even over a single weekend sprint.

sprinting

So, while we were still on the way to PyATL, we came up with a set of two goals for our little experiment and ended up announcing it at the PyATL meeting that night. We wanted to

Things flew together very quckly, and one week later, I flew back down to ATL for a trip to Optio Software, and five of us met up at Jonathan’s house the following morning, for a day of fun filled hacking.

While Jonathan and I worked on hacking out the dispatch piece of RhubarbTart to include in Pylons, Rick Copeland quickly re-implemented cherypy style dispatch in a new, more flexible way. Soon we were able to convert a new pylons project to the TG style dispatch, setup a simple default route, and go.

Our next job was to get the @expose style decorator working in turbogears. Our plan was to push the work of actually creating the correct responses down into the base controller class, and just use @expose to register attributes on the function. That way the decorators could do just one thing — registering how to respond to particular requests. Our new @expose decorators look the same as the old ones, but are much, much simpler, and it will be much easier to extend them in the future.

Pylons buffet support helped out greatly, because we were able to get Genshi, and Kid working right out of the box. And the very first time we tested automatic JSONification via TurboJSON everything worked immediately — since it’s just an Buffet engine. We did have to do a bit of work to make sure that the special Pylons context object was filtered out.

With all that done, we decided it was time for dinner at the Mello Mushroom where they had much needed beer and pizza. Over some BBQ chicken pizza we discussed what still needed to be done. We’d pretty much acomplished our goals for the weekend already. But we had time so we decided we needed support for positional parameters, the input validation, and the @error_handler decorator and a few other things.

Back to Jon’s house we went. Where after much spirited discussion Mike Shnekel eventually convinced us that it might be a good idea to use extension names as part of the content negotiation process. So, a request to turbogears.html would go to the turbogears method, with a request for HTML results, and a request to turbogears.json would go to the turbogears method with a request for a JSONified result, and a request to turbogears.txt would ask for a plan text version.

We all agreed that was good enough for one day, and decided it was time for sleep.

We met again at Pannera Bread on Sunday, worked on database issues, got error handling working, talked to Alberto about possible validation implementation ideas. Implemented one way of doing things, while Alberto implemented another, and we finally resolved the issue in a better way than any of us had thought of before we came.

We also created a paster template for creating a new TurboGears project, in much the same way that you could with tg-admi quickstart. Noah Gift made a version of tg-admin that does exactly what you’d expect it to. The other tg-admin tools like tg-admin sql create are not yet ported, but Noah says he will continue to work on the.

Oops, as some folks were leaving for the night, I discovered the Pannera in Dunwoody closes at 7pm. And I was tired anyway, so I decided to call it quits for the night.

We definitely got more done than we thought we would.

Our littleexperiment was an unqualified success. We were able to get the core TurboGears API working very quickly, with very little code, and even to add a couple of new features as we went.

I think experimental code like this is one of the best reasons to get together and sprint. It’s a lot of fun, the energy is high, and you can really test out an idea quickly.

And I think the results of our experiment will be extra useful because we made intentional implementation choices to keep our new TurboGears layer compatible with regular Pylons, so you should be able to mix and match TG style controllers with Pylons style controllers and use Routes along side TG style Object dispatch. Very cool.

Thanks everybody who showed up.

Since the sprint, we’ve been talking about how this impacts our plans for the future of TurboGears. But, more about that soon, hopefully tommorow. But I’ve got a bit more hacking to do tonight. :)

Changing Values

It’s hard to change your corporate culture, because it requires changing the way people think, feel, and act. And smart people can do really stupid things when they try to change other people.

Once, long ago, I worked with a company that was struggling to attract top notch people in a very small field. The were constrained by the number of people they had, and the were loosing good people, who were hard to replace. So, they wisely decided that they needed to to change their corporate culture, and to make themselves a more attractive place to work.

Their primary tactic in achieving this strategy was to publish a new “company values” statement, and adding “cool” to the list of criteria by which employees were judged at review time. Figuring that if their employees became cooler, people would be more attracted to working with the company. At the same time they also hosted “mandatory fun” activities outside of work hours, on top of the mandatory overtime, and near-constant travel, which was the norm for many of their employees. Head In Sand

At the same time the manager, who was the force behind this, continued to speak condescendingly towards others he perceived as “less valuable” in the organization and was well know for his political games, and his “me first” attitude. This lead to widespread fear that the subjectivity of the “cool” rating was just an attempt to provide material for weeding out the “undesirable” employees.

Needless to say, their attempts did not earn them a worldwide reputation as “one of the most fun places to work.” And many people who felt their contributions weren’t valued left, and they told stories which made it even harder to find people in their small field.

If you want to change your corporate culture, you have to change the way people think, the way they feel, and the way they behave.

Changing people is hard. Rearranging words on paper is easy. Smart companies don’t confuse the two.

If you want to effect real change in an organization:

  • Start with changing yourself. You are part of the culture, and if you are a leader people are watching what you do more closely than what you say.
  • Be honest with yourself and with everybody involved about the problem. Take responsibility for your part what’s gone wrong.
  • Ask people to join you in the hard work of making things better.
  • Continually look for the reasons behind people’s resistance. Perhaps there’s a lack of trust, perhaps the your words are saying one things, and your actions are saying another.

Honesty, openness, and ability to communicate a vision of how things could be better will get you a long way. But you also need to be prepared for resistance, and you need to learn from that resistance. As Jerry Weinberg says:

“Overcoming” is not what you want to do with so-called resistance. What you’re calling “resistance” is what it looks like to you when [people] don’t feel safe following your suggestions. So, what you want to do is learn from it–it’s a gold mine of information, as long as you don’t push to “overcome” it.

Another thing you might want to consider in that most negative corporate cultures trickle down from the top. Contempt is the sulfuric acid of organizational change, it creates defensiveness, super-charges resistance, and corrodes working relationships.

So, if your managerial team shows any traces of contempt, you might want to do is buy a couple copies of Bob Sutton’s new book “The No Asshole Rule” and slip them on a few key people’s desks.