The first thing we did was create some new pages for downloads. Recently we releases a new service designed just for open source project leaders who want to use sf.net as a directory and downloads service.

But, we’re also aware that one of the most important services we provide is project hosting. For the last several months a small group of us have been trying to bring sourceforge.net’s tools into 2010. And now we’re releasing an early preview of those new developer/community tools:

We have a long way still to go, but every long journey begins with a single step, and today’s step is allowing you to try the new forge, to create new projects at:

https://sourceforge.net/register

Where you can go to get a new project, with our new tracker, wiki, git, svn, and other tools. Projects can have subprojects, and links to other tools hosted off site, along with the many features that sf.net brings (free web hosting, hosted apps, etc).

But, why do all this?

In 1999 SourceForge was cool.

It provided all the tools that an open source project needed to get going, from cvs hosting, to bug tracking, and e-mail list support.

They pioneered free free software project hosting, and helped to transform the software development culture from one which barely new about free software or open source, to one where nearly everybody I know uses open license software. Oh sure, some of them might not know it, but they have it on their phones, in their TVs, their wireless routers — not to mention all the websites they use everyday that run on open source.

But, time passed.

More alternatives came out, more projects (including my own) started self hosting, and the landscape of open source software development changed. SourceForge.net took a long time coming out with support for new tools like svn, and then git.

Still, SourceForge has a special place in my heart. Partly it’s nostalgia, I suppose, but I still think:

• the core mission is still right
• and there is still a real need

We (Open Source developers) still need tools like git, mercurial, and svn hosting. We still need bug trackers and mailing lists. And in a meeting of other open source project leaders last fall, nearly every single one of them identified the time wasted integrating and administering these tools as one of their most important frustrations.

Not enough…

But, for many sourceforge.net and other free project hosting services were just not good enough, they weren’t scriptable, the weren’t extensible, their data wasn’t portable, and so they felt like they had to take on that cost.

And I fundamentally believe that open source projects live an die by communication, and that sourceforge.net can do something new by integrating the various kinds of “conversations” that happen around the project. We can integrate mailing lists and forums, we can integrate SCM and ticket trackers, etc.

New and improved

So, a couple of us have been quietly working on something new. The new forge is designed around a few core ideas:

• that data should be portable (every project gets their own database, which they can take with them if they want),
• that the open source community ought to be able to extend and enhance the tools they need,
• that integrating and cross linking the various kinds of conversations that open source projects need to have ought to be easier.

So, what we’re announcing today is more of a commitment to getting there on all these things, and a commitment to the “release early, release often” project management strategy.

So, expect us to take your feedback and make things better. Expect us to release lots of small fixes, and expect a few places where things are broken/incomplete because we value feedback more than polish at this point.

Meanwhile Gerry Weinberg, who wrote several books that I love, and gives lots of great advice, including the some of the best advice I’ve ever read about how to give advice, says “every problem is a people problem.”

So, which is it?

Are bad things that happen the result of bad processes, are they the result of things people do?

I’ve been party to a bit of discussion about this in the last month or two, and in the end it’s all pretty silly.

Processes are created by people, implemented by people, and are designed to accomplish the goals of people.

People run processes!

So, whenever something is broken, it’s people who will need to find the problem and fix it.

People can and do think of ways to improve processes everyday, but I’ll eat my shoe if you find a process that thinks of a way to improve people.

But there’s still a HUGE problem.

Modern companies seem to have a persistent failing — they look for people to blame when something goes wrong — and ignore the context in which those problems happened.

When something goes wrong, fire some people, and replace them with new people who make the same mistakes all over again.

Sometimes you “get lucky”.

The company might get lucky and find a person who’s able to raise awareness, reveal the larger contextual problems, and succeeded in spite of the fact that everything’s stacked against her.

More often than not though, the poor new guy doesn’t see the systematic pressures that caused everything to fall apart, at least not until it’s too late.

Sometimes replacing what’s broken isn’t enough.

Sometimes it’s the equivalent of a mechanic replacing your car’s engine several times in a row, because it keeps burning up — without ever checking to make sure oil is flowing normally, and the cooling system is working.

The easy way out.

It’s often easier to blame people because they don’t “control” them they way they do the context. This blame game is as old as the hills, but definitely not as pretty.

Help people fix processes

The solution is to ask people to look for the systematic pressures, give them the tools to find them, and to empower them to change the way work gets done.

In the end, people will improve the processes, if they believe they are allowed.

Sometimes a design isn’t working because you think you can’t change the one element that needs to be changed.

–Ryan (via svn)

The same thing is true when you are designing the processes by which work gets done.

It’s all about users.

The big user experience wins come from some heuristics that we’ve developed to guess the best file to download. We’re analyzing your browser’s user agent for information about your operating system. And when the project has not told us about their preferred download, we’re analyzing file names and other data to get our best guess as to which file is the most relevant for a particular operating system.

This means we can generally give you a direct link to the right version of the file, right on the project page, so there’s no need to browse through a complex array of pages, and links just to finally get to the file you came to download in the first place.

Now with TG2 goodness.

Under the hood, there’s a new TurboGears 2 app powered by MongoDB. We’ve put this all together very quickly, and there have been a couple of rough spots here and there. Fortunately, none of the rough spots were in the TG2 or the TG2 stack.

But we did have a couple of rough patches. For example, we couldn’t use a mongo replica pair for the master database and have slaves on each node. So, we chose to try running the site all against a single replica pair rather than to do master-slave everywhere. This was compounded with a coding issue that, we were trying to pull tons more data out of MongoDB than we needed, and we ended up saturating a 2 gigabit network connection between the mongo master and the local slaves. Which if you think about it is kind of amazing, since MongoDB was still only using a few percent of the CPU on the box.

At this point though, all those issues are resolved and we are very confident that we’ll be able to add more user-facing improvements to SourceForge project pages and make the download side of things even better.

ArbCamp will be tomorrow night, in the upstairs of the downtown Cottage Inn’s, so this is kind of last minute, but I think it’ll be a very cool event. It’s an UnConference, and people will be self-organizing a variety of sessions, and the possibilities are endless.

One of my main goals when looking for this job was to make sure that whoever I worked for was committed to growing and takeing care of the TG development community. And I couldn’t have asked for anything better. Working for Predictix will help me to polish up the good work that’s already been done to get us to a TG2 beta release, and they already have a lot of fantastic stuff that they would like to open source, which I’m really excited about.

TG2 is moving forward like crazy. In the last three weeks, we’ve had two sprints, both of which had several people working on docs, and on adding the last few features needed for the beta, and cleaning up the show-stopper bugs in our ticket system.

I’m a bit burned out by all the activity, but at the same time I’m very excited about where we are going. I think 2008 is shaping up to be a really busy year for the TG dev team. I see my job in the very short term as creating some stability and consistancy in the midst of the firestorm of new development that’s going on. So, my highest priority right now is getting a stable beta release out the door, and helping us to move forward the docs so that anybody who wants to try out TG2 has a stable base to work on.

My plan will be to do releases about once a month for the rest of the year (or until we have a TG2 final release), because there’s a lot going on, and I want to make that stuff available to people as soon as possible.

Python get’s a very positive review:

Today you’d be nuts not to look seriously at PHP, Python, and Ruby.

So, the rise of the so-called scripting languages is one of the inflection points, but it’s not the only one.

He singles out web-framework development as one place where there’s a lot of stuff happening, and a lot of new “rails-like” frameworks are cropping up all the time. TurboGears will live or die in the context of a much larger web-development revolution, and we need to be prepared to make our way forward in the midst of that.

What comes after rails will not be a rails clone. It will learn the right lessons from rails, avoid the pitfalls of rails, but it will also need to carve out something new and better than rails. For RDBMS users, I think the key difference between TG and Rails is the power and flexibility of SQLAlchemy. We need to “sell” this better.

There are a lot of other revolutions coming according to Tim. And I do think we’re looking at big changes in terms of everything from programming language choice, to web-development tools, to end-user desktops, and data persistence mechanisms. We’re also just beginning to see what the world of high-end javascript and other “rich” internet applications is going to do to our view of end-user software.

He doesn’t even mention the rise of EC2 and the Google App Engine as sea-changes in the way we buy computational resources, and I think that’s going to have a huge impact.

In the end my prediction is that the way we develop applications will change more in the next 5 years than it did in the last 5, and it’s time to start getting our heads wrapped around these issues, or we’ll be left behind.

They are doing some fantastic things with Python in the One Laptop Per Child project.

They have a very cool TurboGears application that will radically improve the transparency of the way out legal system handles “intellectual property.” And they plan to make orders of magnitude more intellectual property data publicly available than we’ve ever seen, and they intend to do it with a python/TurboGears based system.

Ultimately I think this data has the potential to:

• to expose people who abuse patent, copyright, and trade secret laws,
• and to highlight problems with the way intellectual property laws are enforced,
• and ultimately to give us the raw data we need to make intelligent and lasting changes in the whole system.

At the same time it can help to prove that you can build high-traffic, robust systems with Python and TurboGears. They are also doing all kinds of interesting things with search, language processing, and have some amazingly smart people on their team. But they need a fantastic web developer to help them get this amazing stuff into the hands of real people soon.

Which is where you come in, if you’re available or can make yourself available, and you want to be part of something really impressive, feel free to drop me an e-mail at compoundthinking@gmail.com, and I’ll put you in contact with the right people.

If anybody is willing to show up in Ann Arbor, I can provide food, a place to hack, and some good clean post-sprint debauchery. For, unfortunately those sprinting outside of the Ann Arbor area will have to provide their own space, food, and most of all their own debauchery.

If you’re interested in participating in the sprint, please join the Sprint Coordination mailing list.

We’ve got folks committed to:

• Migrating test infrastructure to CherryPy 3
• Finalizing the TurboGears 1.1 configuration system, and implementing what we decide on
• Workiing on the turbogears SQLAlchemy integration to make it easier to use multiple databases
• Doing some general clean up of open tickets

My personal goal is to help get the trunk into a usable state, which means tackling the configuration issue. Once people can run their existing projects on the trunk, TG developers (like me) will be able to do our daily work on turbogears projects on the trunk. Which in turn should help us to attract more developers and testers and continue to make TurboGears a great tool for rapid web application development.

But I really want to start running some online classes again.

I’d like to help people who are facing the learning curve of Web 2.0. Perhaps you’ve done some basic web stuff, but the thought of HTML+CSS+JavaScript+ServerFramework seems like it’s too much to learn on your own. My theory is that it’s better to learn in groups, so what I hope to do is organize some online classes which bring together groups of people who are interested in helping each other learn all this web 2.0 stuff.

I have some basic materials from the PyCon 2007 tutorials on TurboGears, which should supliment the book to provide plenty of material for the server side stuff. And we can use some existing resources for the HTML, CSS and JavaScript stuff.

I’d also be interested in doing a local class in the South East Michigan area on a couple successive Saturdays — but I need a good location, so if you’re in the area, and want to help make this happen drop me an e-mail at (mark at compound thinking dot com).

I’m still getting getting the website setup for all of this, and how I’ll be able to manage the bandwidth requirements of the class. An the class will be “free,” except that I’m going to ask people to pledge to volunteer some time on either a local charity, or working on some open source project.

This will be a learning experience for me too, so if you’ve done something like this before, feel free to e-mail me with whatever tips you might have (mark dot ramm at gmail dot com).

And if you’re interested in taking the class, watch hear for final details sometime next week.

Of course, we’ll be hosting this in Ann Arbor, but we hope it will be a larger regional event. And if you’re interested in helping, feel free to email me (mark dot ramm at gmail dot com) and I’ll get you hooked up with the rest of the interested parties, and we can start rockin’.

If you’re interested, or want to help out, drop me a comment or e-mail me at mark at compound thinking.com

Having a book is good for TurboGears, but we couldn’t cover everything in the book, and not everybody wants or can afford dead-tree documentation, so it’s also important that we create great online docs!

Date: Saturday, 13th January 2007
Location: Ann Arbor, MI, United States (remote participation encouraged!)

We’ll discuss our work on the TurboGears Docs mailing list, on IRC channel #turbogears on freenode, and organize our efforts around the tasks listed below.

Things to work on:

• Widgets Documentation
• Updated “Big Picture” Diagram
• Creating Diagrams to go with various doc pages
• Complete migration of pages from the old trac wiki to this wiki
• Updating DocStrings in the TG code
• Workign on automated API Doc generation
• Creating a script to generate flat HTML versions of the Docs for offine use

There will be work for everybody, at every skill level. We need people to copy over the text from the trac wiki and convert it into ReStructured text, and we need people to update the DocStrings in the TurboGears code. If you don’t want to write docs, but feel comfortable hacking up some code, we need help generating API docs, and we need to create flat HTML files so you can browse the docs offline.

So, if you have a few hours and want to help TurboGears have great Documentation, please pitch in and give us a hand. It will make all the difference in the world for somebody just getting started!

Looks like Python and Ruby are both going to have a good number of talks. I’ll be talking about SQLAlchemy, which is the best object relational mapper I’ve ever seen. There’ll be talks about Test Driven Development in Python, Enterprise Architectural Patterns for Python developers, along with lots of cool talks about Lean Software Development, the side benefits of Test Driven development.

You can still submit a talk proposal before November 30th, and you’ll get free room and board. I think it would be great to see somebody talk about Dabo and Desktop application Development in Python, and they seem to be missing any talk about OSX/Cocoa stuff, which I’m sure is because they haven’t had any proposals yet.

It would also be nice to see a good cross platform development with Mono talk…

I’m really excited by the opportunity to get developers of all kinds together and talk about how to be productive and learn more about the strengths and weaknesses of the various tools/frameworks people are using.

I’m definitely tired, and it seemed like everything that could break at work broke. Then I got home and was working when the TV in the living room made a loud popping noise and started smelling like burned electrical components.

But then I saw Kevin’s post that the book we just published is the 21st most popular technical book on Amazon. I checked out our sales rank, and we’ve also cracked the top 1000 in all books. That means we’re more popular than any other Web Framework, Python, or Java book, and more popular than Tom Clancy novels.

I know it won’t last, but it does make the day better. :) Thanks everybody!