November 28th, 2009 by Mark Ramm
I wrote the first draft of this a long time ago, and I skipped it because tempers seemed high in some places. It seems like things have calmed down, and I think the points are still 100% relevant.
For those reading this remember, I’ve used Django to build things like http://fossfor.us and I have done enough Django surgery to know what I’m talking about.
Django Developers have said, over and over again:
Django is tightly integrated and loosely coupled.
On the face of it, this statement is paradoxical (more techncally, it sounds like an antinomy to me) coupling and integration are generally seen as ends on a spectrum. Perhaps the Django catchphrase is more than just marketing speak, and actually means something.
If so perhaps they are thinking that they are more loosely coupled than some, and more tightly integrated than others. I suppose you could put opinionated frameworks like Ruby on Rails on one end, and free-form frameworks like Pylons on the other, and Django sits somewhere in between.
But even that seems an oversimplification, Rails is a huge community and Rails users have lots of options, there are alternative template engines galore, and many other components have plugins which replace or seriously modify their behavior.
Lest you think I’m manufacturing this from the air, here’s a quote from a django proponent discussing the idea of loose coupling in Django:
Developers coming from Ruby on Rails or other extremely opinionated frameworks may be used to following their framework’s best practices to avoid fighting against a framework which feels that it knows your project better than you do, but with Django you’ll be back in the driver’s seat.
– Will Larson on “loose coupling” in django
I think this is only true, if it’s true, by degree. It’s easier to do radical surgery to the framework in Django than it is in Struts, though I’m not all that convinced that Rails is harder to change than Django. But really, that’s beside the point. The fact of the matter is that framework surgery is much harder to do in Django than Pylons, and I think that’s not a wild claim, but a verifiable fact.
I wouldn’t recommend decoupling from the Django ORM without an extremely compelling reason. It is the most coupled of all the subparts of Django, and certainly not trivial to replace.
– Will Larson on using SQLAlchemy with Django (emphasis added)
Using SQLAlchemy in Django is not really hard (but at the same time it’s not all that easy either). But, it’s the kind of thing you wouldn’t want to do unless you really had a good reason.
Why? Because there are an awful lot of Django components that are “tightly integrated” with the django ORM.
But before everybody gets too mad at me, let me turn it around, and say that I think:
- it is not a bad thing to value integration above coupling.
Another Quote from Will’s blog, shows that he’s aware that loose coupling isn’t the single core value of Django:
Django places value on loose coupling, but it isn’t the sole design principle either.
– Will Larson (in a comment here)
I would say that even more strongly:
Django should not place loose coupling above developer productivity.
Adrian made a good point after my talk by suggesting that developers need to get things done, and to make sites that work now — not create software that is perfect by some abstract standard of design.
And I do think that some of the helpers that depend on the Django ORM are significant productivity wins. And, removing the django ORM dependency in all of them would be both very hard, and totally not worth it.
“Tightly Integrated” has value, and sometimes that value trumps “Loosely Coupled.” Zope is pretty tightly integrated into the ZODB. Many TurboGears 2 plugins are going to be pretty tightly integrated into SQLALchemy. Others will not, and I have encouraged some folks to rewrite their TG2 plugins to make them into pure WSGI apps that don’t depend on TG2 at all. Determining what exactly you will depend on, and how tightly you will be coupled to that dependency requires thought, and ultimately has consequences.
I’m hard on the Django folks here because I think the “tightly integrated/loosely coupled” buzz phrase is actually detrimental to understanding how the trade-off’s work.
And there are trade-offs and those trade-offs mean that there isn’t and will never be one perfect web-framework which somehow magically isn’t subject to the down-side of any of the constraints and design trade-offs that we all have to deal with every day.
Which brings me to the other major point I tried to make in my talk about django. Encapsulation, orthogality, or loose coupling is to some extent enforced by package boundaries. It’s not so much that you can’t be tightly coupled to the internals of an outside package, but that it feels wrong. And it feels wrong because when you start making libraries you start defining public interfaces, and making decisions about what’s internal and what’s external. And that makes monkeying with the internals feel icky. But if everything is all in one package, it’s a lot easier and less ‘icky” feeling to just grab some internal bit and do what you need to, since less thought has been given to what’s public and what’s private.
July 18th, 2009 by Mark Ramm
My last post might have left some folks thinking that we’ve been focused entirely on “end user” experience at SourceForge and have been ignoring the developer side of the equasion, if that’s you, and you’ve felt a bit left out, there’s very good news. In the 7 months since I’ve been here, there have been a significant number of developer focused changes at sf.net, and there are more coming before OSCON.
Hosted Apps
One of the biggest changes is the new Hosted Apps system. We’re an Open Source hosting company, and we want to provide some of the best tools available for Open Source developers, so it only makes sense to use Open Source tools to do it. So, we now provide two dozen applications that you can install and use to help develop and manage your project, including trac, mediawiki, dotproject, a microbloging system, phpbb forums, an app for brainstorming ideas, and lots more.
From the perspective of an open source project maintainer, I think the best part of this is that I don’t have to manage them, do upgrades, backups, or worry about downtime — there are other people responsible for all that.
Have it your way
Part of the plan here is to make SourceForge more modular, and to let project managers use the tools that make sense for them and for their project. I think we’re the only open source hosting solution that provides svn, git, hg, bzr, and cvs source repositories. And with Trac, you’ve definitely got a far more full featured bug tracker than is available in most other open source project hosting.
One of the other advantages is that if you want new features in SourceForge, there’s now a clear and obvious way to do it. For example if you want some new ticket tracking feature you can add that feature to Trac, and once there’s a new release, it’ll be added to your sf.net project for free.
How the Consume side fits in
If you saw yesterday’s post, you’ll remember this diagram.
We’ve created the summary page based on the idea that projects have data in various places, in SourceForge developed apps, in hosted apps, on freshmeat, and out on the web in various places. In fact, all the the data on the sf.net project summary pages and download pages is fetched from the existing php apps via feeds and public API’s.
We still have lots of work to do, but all of this means that we’re changing the way SourceForge works to serve our developers better. We’re giving up on the assumption that we can provide the one right set of tools for all open source projects, and we’re also trying to leverage and improve existing open source solutions rather than reinvent the wheel.
To be fair, lots of this stuff wasn’t available when sourceforge got started, so back then we had to do some inventing, but the open solutions have passed us in lots of areas, and we’re taking advantage of that.
Developers need users too
And that brings us full circle, we want to grow project communities. In the end this means serving two sets of people well, and it means bringing those two communities together, serving developers means they get the tools they need to make better software, and serving downloaders and end users means the overall community grows. But the real growth happens when the line between developer and user begins to blur, with non-developers triaging bugs, writing documentation, doing translations, and sometimes even becoming developers themselves.
June 22nd, 2009 by Florent Aide
Since the beginning (nearly) of TurboGears 2.x development we use a private index to store all our dependencies and point people to it for installation. This was nice because it helped us control our dependency tree and make sure installation of our unstable software was as easy as possible.
Since we had released a stable 2.0 version, followed by a 2.0.1 bugfix, we really were lacking a real pypi install mechanism. This is now a thing of the past! You can install a full blown TurboGears2 environment in one simple command:
easy_install tg.devtools
and this will fetch everything using normal lookups from the pypi. I hope this will help people out there.
At the same time I’d like to ask for help in that domain: If you are using TurboGears2 and find out that some dependency that needs compilation is not found in a compiled state (either on pypi or on our index), please help us:
- By sending a compiled version of the egg to me (florent aide, you’ll find me on the mailing list easily)
- By pointing us the missing dependency (send a mail in the turbogears or turbogears-trunk mailing list)
- Or even better: try to become a maintainer for the specific dependency, that you know will always be missing for your architecture, and let us know about that great news. The whole open source community will become better, your karma (the real one, not the one you can admire on ohloh) will bump-up and you’ll be forever remembered on endor.
Now this leaves us with one more thing to do: finish the web site for turbogears that is currently in a poor state. If you would like to help us on the engine, please let us know in the mailing lists or via private message. If you want to help-out with content, please let us know also and we’ll be more than glad to give you an editor account and some directions to get some content online in this new website.
Cheers all,
Florent.
June 15th, 2009 by Florent Aide
In the last post I said we presented TurboGears 2.0 at the Pycon FR event. We have received the video feeds from the organization. This is a French presentation, sorry for the non-french speakers.
As a reminder the code of application we presented is available on bitbucket.
June 12th, 2009 by Florent Aide
Two weeks ago we (Jonathan Schemoul, Thomas Zaffran and me) animated a TurboGears 2.0 presentation (in french) and workshop at PyconFR 2009. PyconFR is an annual French Python event organized by the AFPY.
The presention was separated in two phases. The first part covered an history of the project and a technical overview. The second part was a live coding session during which we created a small todolist application to show-off the new TurboGears 2.0 features.
At the end of this presentation, one of the attendees came to us and gave us a nice remark that I would like to share with you here: “I did not remark any changes in TG2 since my TG1 days, did you change something?”
This was a pleasant surprise to me. This means one thing: we changed the whole underlying layers, rewrote authentication and authorization, switched tgwidgets with ToscaWidgets, based our Framework on Pylons instead of CherryPy — and yet this person, looking at the controller’s code of our todolist application felt “at home like if it was a TurboGears 1.0 application”.
As we already stated, we want to ease-out the tg1–>tg2 transitions as much as possible. We know there is some room for improvement in that quarter! But since we (core devs) have live and running tg1 applications out there the itch is already there and we’ll make sure to scratch it.
As an example, Paris Envies is a TurboGears 1.1 application, coming from an 1.0 background, using Genshi, SQLObject and TGWidgets. And I know for sure that Jon has a real incentive to move to TG2 and get SO support for it :)
Stay tuned people, TurboGears is live and well!