Compound Thinking

Ignoring “technical debt” is like playing with dynamite

Earlier today, I posted the somewhat controversial Focusing on Technical Debt is a dead end and apparently I missed some of the subtlety required to get my actual point across.

I still don’t believe that it makes sense to focus on technical debt.

Technical debt is a fact of life, and it’s a sign that you’re writing better code now than you did before. It’s a sign that the team is learning, and that things are getting better. So, you should learn to live with it, or even to celebrate it.

But, that does not mean you should ignore technical debt either.

Technical debt can have real costs, can create real project risk, and most importantly can limit the capacity of the system to produce value. If it’s doing that, you need to fix it. But even then you are better to FOCUS on the long view of increasing the capacity of the system. In other words:

• Make sure your whole team is learning from the process
• Make sure you fix the things that impede progress rather than just things that bug you
• Make sure that you do enough root cause analysis to know why the obstructions got there in the first place

In other words elevate the priority of learning above the need to “fix” things now. That way you’ll avoid future mistakes, get more done, and at the same time continue to deliver value to customers.

In the end I don’t like the term “Technical Debt” as a catch-all because it’s a way of lumping all kinds of different code problems together into one basket which results in trivium being treated with the same seriousness as potentially project destroying technical risks.

So, you do need to pay attention to all technical debt, but some of it should be ignored, some of it should be responded to immediately, and some of it should be “fixed” when you next work on code that touches that thing. Not fixing the critical things is dangerous, and I thought everybody knew that. My point in my last post was that fixing the non-critical things with the same urgency is more insidious danger.

Focusing on it exclusively leads to a weird situation where you’re fixated on the past, and not moving forward anymore. And ultimately it short-circuits the critical thing, which is growing the capacity of the team to deliver value.

The tech of the new SourceForge

Last week I blogged about the new SourceForge.net and one of the first questions I got was when are we going to “lift the covers” and show off our new tech.

There’s definitely more to come in terms of releases and code, but I thought it’d be worthwhile to start with a quick run through of the tech stack and a bit of a description of what we’re doing.

Our first rule for libraries and tools on the new forge, was that we needed to use open source everywhere. Partly this is just because having the freedom to look at the code and modify it where we need fixes, makes it’s the easiest and best way to develop software. Partly it’s because we’re an open source code hosting platform, and we want to use what we promote. But perhaps most importantly, it means that we’re not prevented from sharing our work with others, or from inviting others to work with us in the future.

At the same time we had a company wide decision to standardize on the technology stack that we’d used in the “consume” project last year. So, we’re using:

• Python,
• TurboGears,
• MongoDB,
• and AMQP (RabbitMQ).

The combination of these means that we have:

• a huge number of libraries available to us,
• a web framework that we can turn into a plugin framework for projects and the tools they want,
• a schema-free database that lets us easily version documents to keep history on wiki pages, tickets, and other “artifacts” within the new forge
• a scalable system for handling asynchronous tasks, and propagating update notifications

The choice to use Python has been particularly valuable, since there are (literally) dozens of libraries that we were able to use to help us with everything from encrypted cookie sessions, and mongodb drivers, to markdown text processing, and syntax highlighting.

We’re still in the early days and have a lot more to do, but the goal is an open extensible, system that supports open source projects, and ultimately encourages more people do download and use a wider variety of open source applications.