Archive for the 'TurboGears' Category

TurboGears Joins the Pylons Project

After much debate, discussion, and contemplation, we’ve made an important decision, that will best ensure the future of TurboGears, and of the ideas on which it was based. TurboGears is merging into the Pylons Project.

A bit of background

We built TurboGears 2 on top of a the Pylons framework, and I have been working with Ben and the Pylons folks for a couple of years now.

Pylons is a lightweight framework that is neither full stack noropinionated. The Pylons team has recently joined up with the repoze.bfg folks to create a new low-level non-opinionated web application development library called Pyramid which is based on repoze.bfg and now part of the larger Pylons Project.

Pyramid status

You can use Pyramid to build web applications right now. It has more plug points, and is in many ways more flexible and extendable than the original Pylons and will provide an even better foundation for a full stack framework like TurboGears.

Pyramid has a strong, highly documented, well tested, approach, is already starting to show the fruits of merging in ideas from TurboGears, and Pylons. We expect it to be a great choice for those who want a flexible, non-opinionated and very fast framework for their projects.

The future of “Full Stack” frameworks

Fortunately, that’s not where the story ends. The Pylons Project leaders recognize that a “low level” framework is not enough.

Most web developers need to get things done quickly, and want a full stack setup and ready to go, so they can immediately start developing features rather than infrastructure.

That’s where we come in. TurboGears was the pioneer of the “full stack” set of integrated components approach among the modern Python web frameworks, and we have already developed many full stack tools.

Next Steps

So, our first step will be to add the TurboGears2 package to the legacy support in the Pylons Project. So, the Pylons 1.x and Turbogears2 packages will be maintained side by side as part of a single overall project. This change won’t impact the TG2 code, except that we will be officially acknowledging that both codebases are
tightly linked and now are part of a single project.

Maintaining the existing Pylons1+tg code will only be one part of the larger Pylons Project.

Big Harry Audacious Goal

Ultimately, we will also be working with the Pylons Project folks to create a new generation of rapid application development tools on top of Pyramid, using the TurboGears “Full Stack” philosophy. We will help to pick default templating engines, default session support, default data persistence mechanisms, integrate widget libraries and build high level tools like OAuth or OpenID support.

The main benifits of this merger will come when we reach across framework boundaries, work together with with Repoze, Pylons, and other web framework developers to build a set of high level tools that make building complex, modern web applications easier and faster. There’s a lot we can learn from each-other, and even more that we can do if we work through our differences and find new ways to collaborate.

There is no future but what we make

It’s been a great ride. And, it looks like just when we thought things were settling down, there’s another drop, and the roller coaster ride that is TurboGears isn’t done yet. I am forever grateful for the chance
to work with all the TurboGears developers, to face challenges, and to build something that’s been so valuable to so many. And I’m looking forward to what we can do together with the Pylons and Repoze folks.

Technical Debt isn’t always Debt

After yesterday’s posts about why you should not focus on reducing technical debt and why that’s not an excuse to ignore it either. Dave brought up a good point in a comment.

Technical debt can be assessed like real debt, to a large degree. How much are you paying monthly because of the debt? An annoying little thing, or wanting to upgrade to a new version just for newness’ sake has a low interest rate.

– Dave Brondsema

But it got me thinking, perhaps “debt” isn’t the right metaphor for this at all. Annoying issues that don’t cost anything, and don’t technically need to be “repaid” aren’t properly debt at all.

Perhaps technical warts would be a better term for those kind of things. It feels good to fix them. But rather than feel righteous for spending time fixing them all and “paying down our debt,” perhaps we ought to feel a little bit self indulgent for spending time and money on what amounts to cosmetic surgery for our code.

I’m not saying cosmetic surgery is never valuable, or that warts are never painful, and certainly not that you should ignore them. After all warts can be cancerous, and could perhaps kill you. But after investigating and discovering that they are benign, perhaps it’s not worth the cost to have them removed.

Ignoring “technical debt” is like playing with dynamite

Earlier today, I posted the somewhat controversial Focusing on Technical Debt is a dead end and apparently I missed some of the subtlety required to get my actual point across.

I still don’t believe that it makes sense to focus on technical debt.

Technical debt is a fact of life, and it’s a sign that you’re writing better code now than you did before. It’s a sign that the team is learning, and that things are getting better. So, you should learn to live with it, or even to celebrate it.

But, that does not mean you should ignore technical debt either.

Technical debt can have real costs, can create real project risk, and most importantly can limit the capacity of the system to produce value. If it’s doing that, you need to fix it. But even then you are better to FOCUS on the long view of increasing the capacity of the system. In other words:

  • Make sure your whole team is learning from the process
  • Make sure you fix the things that impede progress rather than just things that bug you
  • Make sure that you do enough root cause analysis to know why the obstructions got there in the first place

In other words elevate the priority of learning above the need to “fix” things now. That way you’ll avoid future mistakes, get more done, and at the same time continue to deliver value to customers.

In the end I don’t like the term “Technical Debt” as a catch-all because it’s a way of lumping all kinds of different code problems together into one basket which results in trivium being treated with the same seriousness as potentially project destroying technical risks.

So, you do need to pay attention to all technical debt, but some of it should be ignored, some of it should be responded to immediately, and some of it should be “fixed” when you next work on code that touches that thing. Not fixing the critical things is dangerous, and I thought everybody knew that. My point in my last post was that fixing the non-critical things with the same urgency is more insidious danger.

Focusing on it exclusively leads to a weird situation where you’re fixated on the past, and not moving forward anymore. And ultimately it short-circuits the critical thing, which is growing the capacity of the team to deliver value.

Focusing on removing “technical debt” is a dead end

Americans, myself included, have a very short term perspective. Results matter, and this quarter’s results matter most. Perhaps next quarter is important too, heck we might even think a year out.

But we rarely take the *long* view. Push harder and you can make this quarter’s numbers a bit better. Push harder and you’ll get this sale in before the end of the year.

Developers do this too. We push harder to get a product out the door on time, push harder to get a feature in to this release rather than the next.

And here’s one which I think is particularly pernicious, because it seems like long term thinking — push harder to reduce “technical debt” right now.

Technical debt is the distance between how you’d write it knowing what you know now, and how you did write it knowing what you knew then. If you’re constantly focused on fixing past mistakes, you’re not learning the things that will turn the code you’re writing today into “technical debt.”

My recomendation is to focus on other things, focus on solving the problems you’re facing today the best way you know how, focus on cleaning as you go, focus on growing the team’s ability to learn — in other words:

Don’t focus on maximizing the utilization of the system,
focus on growing the capacity of the system.

There are a lot of things in projects I’ve written that I wouldn’t do the same way again. But if I get caught up in fixing every bad API decision in TurboGears or whatever, I would be paralyzed and unable to move forward. Of course it’s not hypothetical, I’ve definitely been there, done that. But I’ve learned from that mistake, and hopefully I can maximize the capacity of the system by helping you to avoid it. ;)

The tech of the new SourceForge

Last week I blogged about the new SourceForge.net and one of the first questions I got was when are we going to “lift the covers” and show off our new tech.

There’s definitely more to come in terms of releases and code, but I thought it’d be worthwhile to start with a quick run through of the tech stack and a bit of a description of what we’re doing.

Our first rule for libraries and tools on the new forge, was that we needed to use open source everywhere. Partly this is just because having the freedom to look at the code and modify it where we need fixes, makes it’s the easiest and best way to develop software. Partly it’s because we’re an open source code hosting platform, and we want to use what we promote. But perhaps most importantly, it means that we’re not prevented from sharing our work with others, or from inviting others to work with us in the future.

At the same time we had a company wide decision to standardize on the technology stack that we’d used in the “consume” project last year. So, we’re using:

  • Python,
  • TurboGears,
  • MongoDB,
  • and AMQP (RabbitMQ).

The combination of these means that we have:

  • a huge number of libraries available to us,
  • a web framework that we can turn into a plugin framework for projects and the tools they want,
  • a schema-free database that lets us easily version documents to keep history on wiki pages, tickets, and other “artifacts” within the new forge
  • a scalable system for handling asynchronous tasks, and propagating update notifications

The choice to use Python has been particularly valuable, since there are (literally) dozens of libraries that we were able to use to help us with everything from encrypted cookie sessions, and mongodb drivers, to markdown text processing, and syntax highlighting.

We’re still in the early days and have a lot more to do, but the goal is an open extensible, system that supports open source projects, and ultimately encourages more people do download and use a wider variety of open source applications.