Archive for November, 2009

Power, Authority, Force and the politics of software

Bruce Eckel recently posted “We No Longer Need Power,” and Ian Bicking recently gave a talk “Toward a new self-definition for open source“. Both raise similar points, “power” seems to be handled differently — actually they both say better — in open source communities and open spaces conferences than traditional companies.

I agree, but I think that any discussion power politics is pretty risky, particularly since we use authority, power and force pretty interchangeably in many situations. And it’s hard to nail down exactly what people mean because there’s some confusion even at the level of what the words mean.

So, from my perspective (heavily influenced by Hannah Arendt here’s a reasonable definition of terms:

  • Power: group action in service of some goal
  • Authority: some kind of status “given” to an individual, which gives her influence over the actions of others.
  • Force: the ability to make somebody do something they would not do on their own
  • Violence: the act of causing of physical, social, or economic damage to an individual or group.

( * Arednt fans should note that I am using force differently than she does.)

Power and Authority in Open Source

Open Source projects have powerful people, in that they have people who somehow manage to get a group of people to work in common on some larger goal. And there’s often some sort of formal authority which resides in the BDFL or project leaders. Authority often centers around who has commit access and who can flip the commit bit. But there’s very little force or violence involved, and the authority that we give is always tentative and revokable in the case of an “unfriendly” fork of the community.

The fact that there’s often not much money going on reduces the chance of economic violence and removes the main way that developers encounter force. So, nobody can force you to work on stuff you don’t want to, and nobody can force you to stop working on the things you really want to see done.

Open source is Force/Violence free

At least that’s how it often seems. But I there can be significant social pressures to work on one thing rather than another, and many people often feel like they have a right to tell you what you ought to be working on.

None of this counts as force, but it does seem to indicate some imbalances in the system, where there are producers and consumers of open source and the producers seem to often feel like they can never do enough to please all of their consumers.

And because there’s a lack of resources going to the producers, it’s consumers are often frustrated by the slop pace of development. But all of that is another post for another day.

P.S. A Science Fiction book recommendation

Ursula K. LeGuin wrote a great book that I’ve been thinking a lot about when talking to people about the subject of the difference between “commercial” software culture, and the somewhat anarchistic Open Source culture. The book The Dispossessed, is one of my favorite Science Fiction novels, and and though it came long before “Open Source” or even “Free Software” even existed as such, I think it has a lot to say about the strengths and limitations of the “Free Software” culture.

Coupling Django Style

I wrote the first draft of this a long time ago, and I skipped it because tempers seemed high in some places. It seems like things have calmed down, and I think the points are still 100% relevant.

For those reading this remember, I’ve used Django to build things like and I have done enough Django surgery to know what I’m talking about.

Django Developers have said, over and over again:

Django is tightly integrated and loosely coupled.

On the face of it, this statement is paradoxical (more techncally, it sounds like an antinomy to me) coupling and integration are generally seen as ends on a spectrum. Perhaps the Django catchphrase is more than just marketing speak, and actually means something.

If so perhaps they are thinking that they are more loosely coupled than some, and more tightly integrated than others. I suppose you could put opinionated frameworks like Ruby on Rails on one end, and free-form frameworks like Pylons on the other, and Django sits somewhere in between.

But even that seems an oversimplification, Rails is a huge community and Rails users have lots of options, there are alternative template engines galore, and many other components have plugins which replace or seriously modify their behavior.

Lest you think I’m manufacturing this from the air, here’s a quote from a django proponent discussing the idea of loose coupling in Django:

Developers coming from Ruby on Rails or other extremely opinionated frameworks may be used to following their framework’s best practices to avoid fighting against a framework which feels that it knows your project better than you do, but with Django you’ll be back in the driver’s seat.

– Will Larson on “loose coupling” in django

I think this is only true, if it’s true, by degree. It’s easier to do radical surgery to the framework in Django than it is in Struts, though I’m not all that convinced that Rails is harder to change than Django. But really, that’s beside the point. The fact of the matter is that framework surgery is much harder to do in Django than Pylons, and I think that’s not a wild claim, but a verifiable fact.

I wouldn’t recommend decoupling from the Django ORM without an extremely compelling reason. It is the most coupled of all the subparts of Django, and certainly not trivial to replace.

Will Larson on using SQLAlchemy with Django (emphasis added)

Using SQLAlchemy in Django is not really hard (but at the same time it’s not all that easy either). But, it’s the kind of thing you wouldn’t want to do unless you really had a good reason.

Why? Because there are an awful lot of Django components that are “tightly integrated” with the django ORM.

But before everybody gets too mad at me, let me turn it around, and say that I think:

  • it is not a bad thing to value integration above coupling.

Another Quote from Will’s blog, shows that he’s aware that loose coupling isn’t the single core value of Django:

Django places value on loose coupling, but it isn’t the sole design principle either.

– Will Larson (in a comment here)

I would say that even more strongly:

Django should not place loose coupling above developer productivity.

Adrian made a good point after my talk by suggesting that developers need to get things done, and to make sites that work now — not create software that is perfect by some abstract standard of design.

And I do think that some of the helpers that depend on the Django ORM are significant productivity wins. And, removing the django ORM dependency in all of them would be both very hard, and totally not worth it.

“Tightly Integrated” has value, and sometimes that value trumps “Loosely Coupled.” Zope is pretty tightly integrated into the ZODB. Many TurboGears 2 plugins are going to be pretty tightly integrated into SQLALchemy. Others will not, and I have encouraged some folks to rewrite their TG2 plugins to make them into pure WSGI apps that don’t depend on TG2 at all. Determining what exactly you will depend on, and how tightly you will be coupled to that dependency requires thought, and ultimately has consequences.

I’m hard on the Django folks here because I think the “tightly integrated/loosely coupled” buzz phrase is actually detrimental to understanding how the trade-off’s work.

And there are trade-offs and those trade-offs mean that there isn’t and will never be one perfect web-framework which somehow magically isn’t subject to the down-side of any of the constraints and design trade-offs that we all have to deal with every day.

Which brings me to the other major point I tried to make in my talk about django. Encapsulation, orthogality, or loose coupling is to some extent enforced by package boundaries. It’s not so much that you can’t be tightly coupled to the internals of an outside package, but that it feels wrong. And it feels wrong because when you start making libraries you start defining public interfaces, and making decisions about what’s internal and what’s external. And that makes monkeying with the internals feel icky. But if everything is all in one package, it’s a lot easier and less ‘icky” feeling to just grab some internal bit and do what you need to, since less thought has been given to what’s public and what’s private.

Morality and Software Development Leadership

Power, dominance, and responsibility are hot button issues which hover over and around every action leaders take like ghosts. And those who ignore them — who wield power without thought, who see only the ends, and ignore the means — put their projects at risk.

I think it’s a truism that there is no single barrier to IT project success more powerful than bad management. There are dozens, if not hundreds, of ways in which bad management directly and indirectly decreases productivity, and undermines even the the possibility of success.

Foucault, is the twentieth century master of uncovering hidden power relationships, and anybody who’s read his masterwork Discipline and Punish will learn a variety of tools to uncover hidden power relationships, and these tools can be immensely valuable in unpacking what’s really going on when a situation “just feels wrong.”

It’s hard to quantify, but knowing this stuff can definitely help you avoid political landmines, and create a safer work environment. But, I still haven’t figured out how to make Foucault palatable and understandable to the average IT manager. ;)

But Naked aggression is easy to spot

On the other hand, you don’t need a sophisticated set of tools for unpacking power relationships to see what’s wrong with some teams. For example, I once worked for a manager who held daily motivational meetings, in which he enumerated every small mistake that any of us could make that might make him look bad, and let each of us know in no uncertain terms that we would be “terminated” if we were seen to have made even the smallest mistake. If we didn’t answer the phone with the right phrase, or try to convince customers that they wanted more expensive products than they needed, or failed to live up to any of thirty different (and sometimes contradictory) arbitrary rules, we were told we would be fired instantly.

Doing the wrong thing may pay off now, but it almost always hurts you later.

I left after less than 6 months, by the time a year was up, only 2 of 20 of the original people on the team were left. But, the manager received a commendation for “running a tight ship.” Things were looking good. Then again, a year later he was fired, because his team just couldn’t keep up, and everyone on the team was so busy not screwing up that the had no time or energy left for the important things they were hired to do.

Dominance games, and pure aggressive pressure helped him meet his daily and weekly goals, but killed him in the long run.

Project managers need to learn to wield power with a light touch.

It’s inevitable that there are power-relationships involved in the context of an important project. If you’re doing things right people are passionate about the project, people want to do the right thing, and people don’t always agree about what the right thing is.

It can be tempting to jump into these disagreements and make decisions — and it can speed up progress in the short term. But be careful, just like the manager who made his short term goals, but lost in the long run, you may end up killing people’s passion, and cutting off discussion before critical information is revealed.

Sure, there are times when a strong good push can help people avoid prolonged and useless discussions, but it’s too easy to take advantage power relationships to avoid difficult but important discussions.

There’s lots more to be said on the subject of power relationships in software development, but I think one of the key things we have to understand is that computer nerds participate in power struggles too, they just do it naively and instinctively. And that leaves us easy prey for those who do it with knowledge, talent, and finesse. And those people are out there and in places where 2 years is a long time, they can get away with it for a surprisingly long time.

P.S. I wrote this post in 2007, almost published it in 2008, and am just now getting around to publishing it. So, please don’t think this is about anything happening to me personally right now, but it is something I keep seeing “around town” and something I think we need to understand better if we are going to stop falling prey to those who understand power politics better than we do and wield them more aggressively, and end up ruining many good projects.