Meanwhile Gerry Weinberg, who wrote several books that I love, and gives lots of great advice, including the some of the best advice I’ve ever read about how to give advice, says “every problem is a people problem.”

So, which is it?

Are bad things that happen the result of bad processes, are they the result of things people do?

I’ve been party to a bit of discussion about this in the last month or two, and in the end it’s all pretty silly.

Processes are created by people, implemented by people, and are designed to accomplish the goals of people.

People run processes!

So, whenever something is broken, it’s people who will need to find the problem and fix it.

People can and do think of ways to improve processes everyday, but I’ll eat my shoe if you find a process that thinks of a way to improve people.

But there’s still a HUGE problem.

Modern companies seem to have a persistent failing — they look for people to blame when something goes wrong — and ignore the context in which those problems happened.

When something goes wrong, fire some people, and replace them with new people who make the same mistakes all over again.

Sometimes you “get lucky”.

The company might get lucky and find a person who’s able to raise awareness, reveal the larger contextual problems, and succeeded in spite of the fact that everything’s stacked against her.

More often than not though, the poor new guy doesn’t see the systematic pressures that caused everything to fall apart, at least not until it’s too late.

Sometimes replacing what’s broken isn’t enough.

Sometimes it’s the equivalent of a mechanic replacing your car’s engine several times in a row, because it keeps burning up — without ever checking to make sure oil is flowing normally, and the cooling system is working.

The easy way out.

It’s often easier to blame people because they don’t “control” them they way they do the context. This blame game is as old as the hills, but definitely not as pretty.

Help people fix processes

The solution is to ask people to look for the systematic pressures, give them the tools to find them, and to empower them to change the way work gets done.

In the end, people will improve the processes, if they believe they are allowed.

Sometimes a design isn’t working because you think you can’t change the one element that needs to be changed.

–Ryan (via svn)

The same thing is true when you are designing the processes by which work gets done.

Sometimes we think we know that a piece of code is important so we spend some time optimizing it. And in the end it’s less clear, and less maintainable, and it turns out that our bottlenecks are all elsewhere.

But, sometimes we do know where bottlenecks are going to be, we’ve learned from experience, and we know what needs to be done.

We know that architecture determines performance, and architecture isn’t easily bolted on at the end of the project.

So we have a conundrum. We shouldn’t optimize yet because we don’t know where the bottlenecks will be. We shouldn’t wait to optimize because we can’t easily retrofit a good architecture on a complex system.

Some of the conundrum is only apparent — there’s a difference between architectural problems that need to be set up front, and the kind of low level micro-optimization that obscures more than it helps. But, sometimes these conflicts are real — how do I know if I need a multi-process multi-consumer queue system for PDF generation before we build the system and benchmark it? If you don’t need it, that kind of extra architectural complexity just obscures the bit of code that actually solves the problem.

Solving the problem by going meta

Perhaps the problem really is that we’re dumb and optimize the wrong things at the wrong time. The solution to that problem is to get less dumb. Which means that we ought to spend time optimizing “learning”, both within our project processes, and across projects.

Codifying this learning is what the Patterns of Enterprise Application Architecture book was all about.

And I think it’s great as far as it goes, and if you haven’t read it you should buy it now.

But there are a lot of patterns that I can identify from my last half dozen projects that aren’t covered in PoEAA, so it would be great to see a next generation of books and blog posts that cover the modern architectural trade-offs that you have to make, something that covers some of the paterns of the web.

Scalability via in HTTP, etags, caching, and load balancing (the whole RESTful services argument), networked async processing patterns, etc. Scaling to the public web levels requires a whole different set of architectural principles than scaling to the old “enterprise” levels did, and that knowledge seems very much in flux.

It would be great if it also provided some advice for those of us who’ve moved into what Neil Ford has called the world of the Polyglot Programmer, patterns for coordinating activities across language barriers in a sensible way. That’s part of the nature of modern web systems too.

I haven’t gotten to know many people outside the engineering team in Dexter, but they are great guys.

There’s lots of good stuff happening here, support for bazar, mercurial, git, trac, and other options on SourceForge itself, improved feeds, and other API’s for getting at SF data, etc. But I’m only peripherally aware of all that at the moment because I was hired to work on “totally new stuff” which is written in Python.

What I’m working on

Our first new project is a site called FossFor.Us, and it was the vision for this site, and the team that is working on this and other new stuff, that sold me on the coming to work for Sourceforge. It’s written in Django, and it’s been my first really large Django project, and while the experience has been pretty positive, there have been a number of things that have renewed my commitment to TurboGears development — but that’s a blog post for another day.

The backstory to the FossFor.Us site is that open source project hosting providers (Sourceforge and it’s recent competitors) have traditionally been pulled in two very different directions by two very different sets of users:

• developers of open source software
• and people who just want to.

And that tension has held us back in the past, we have to serve everybody with the same portal, and it ends up not serving either community as well as it should. But since developers are the most vocal users, it’s been the second class of user that’s been most neglected.

These people are just looking to get things done, and don’t care about the “project” part of open source software, they are, at least at first, only interested in the “product.” In many ways the Free and Open Source Software community has not served these people well.

Fossfor.us is in it’s first incarnation an attempt to create a window on the free software world, that’s just about finding and using software. But in a larger sense it’s an attempt to help us as a community to connect with potential users better.

I think connecting FOSS geeks and users is actually important

It’s important because people aren’t aware that there are free options, and are paying for software they can’t afford. There’s a prototypical user (based on a real person) that we talk about a lot, who’s a single mom, has an old laptop, and struggles week to week to pay her bills, but who bought Photoshop, because “that’s how you edit photos.” Her family could have used that money to more productive ends, but because she needed to edit photos, and didn’t know about the free alternatives all those opportunities are just lost.

Of course the same thing is true of small business owners, who could use free software to reduce their “overhead” costs, and actually spend money on creating things people love. Free software has the potential to lubricate the wheels of the economy, encourage entrepreneurial activity, and enrich people’s lives.

All of this is to say I think fossfor.us is a way to serve the world by making the product of all the open source developer’s labor more easily available and more accessible to real people. And when my mom actually used it to find some software a couple weeks ago, I knew we’d done something right.

Data is certainly not like many other assets, it doesn’t depreciate, you can copy it endlessly, and it’s next to impossible to imagine a commodities market for data. Heck copying the data can either increase it’s value (think “The DaVinchi Code”) or decrease it (think passwords). People don’t pay for data as much as they pay for human attention. You can use data to get attention or you can use attention to collate, assimilate, and otherwise transform raw data into useful information, but either way data needs people to understand and interpret it to become valuable.

So at best:

data + human_understanding == value

Bruce Schenier takes it one step further, calling data the pollution of the the information age.

Data Pollution

Data sucks up space, time and human attention. But more than that, data can be parsed, manipulated, and transformed to fit various agendas. And in a world where data about all of us is “owned” by various large corporations, from Amazon, to Google, to Enron, it’s not always clear how that data will be used. Besides which millions of credit card numbers are stolen from various companies who store our data “in good faith.” Data costs money in terms of maintenance, in terms of storage, and in terms of liability. Heck, I know people who work for companies who have an e-mail retention policy — which is really more of a mandatory e-mail deletion policy.

Polluted Data

And that assumes that all that data is verifiable true, and that’s definitely not the case. I sold a car once and the new owner didn’t take it to the DMV to get it registered before his friend drove it without a license and got it impounded. And that showed up on my credit report for years. I have a friend who somehow ended up “deceased” even though she’s still very much alive and well.

All of this is to say that as software developers, IT Mangers, and companies in general need to think a lot more about data, and to invest in some better terms for the various different things we call data.

We need to differentiate between raw data, information, and knowledge. We need to help our customers think about the life cycle of the data they want us to capture. We need to educate people about the costs and benefits associated with keeping data, and ultimately we need to follow the mantra:

Think before you store

And if you’re concerned about privacy, and individual liberty, please take a few min and read Bruce’s article.

Why have motivational meetings?

The right way to use motivational meetings is to reaffirm the purposes of the group, and help people to connect the dots between their individual efforts and the collective goals of the group, and to connect those goals with their own individual aspirations.

Basically, motivating people is easy:

• Give them work that is meaningful to them and to the organization
• Treat them with respect

Treating people with respect includes paying them a fair wage, and not doing any of these things.

Among other things it also means not letting people who aren’t contributing to the common goals of the organization hold back the group by not doing their job.

Research has shown that one of the survey questions most highly correlated with motivation and performance is:

Does my supervisor, or someone at work, seem to care about me as a person?

Which is another way of saying does your boss respect you. At the same time the single highest correlation for any question was:

I get to do what I do best everyday at work.

So, it’s really important to line people’s intrisic skills and internal long-term motivational drivers with the work you ask them to do.

If you’re not doing those two things, motivational meetings are a loss. If you are doing them you can use a meeting to remind people of how their deeper motivations are connected to what they are doing now.

P.S. My info on the top questions and their correlation to performance comes from Gallop research via the very interesting book First Break All the Rules, which is one of the best, and most evidence based, books on managing for exceptional performance I’ve read.

Python get’s a very positive review:

Today you’d be nuts not to look seriously at PHP, Python, and Ruby.

So, the rise of the so-called scripting languages is one of the inflection points, but it’s not the only one.

He singles out web-framework development as one place where there’s a lot of stuff happening, and a lot of new “rails-like” frameworks are cropping up all the time. TurboGears will live or die in the context of a much larger web-development revolution, and we need to be prepared to make our way forward in the midst of that.

What comes after rails will not be a rails clone. It will learn the right lessons from rails, avoid the pitfalls of rails, but it will also need to carve out something new and better than rails. For RDBMS users, I think the key difference between TG and Rails is the power and flexibility of SQLAlchemy. We need to “sell” this better.

There are a lot of other revolutions coming according to Tim. And I do think we’re looking at big changes in terms of everything from programming language choice, to web-development tools, to end-user desktops, and data persistence mechanisms. We’re also just beginning to see what the world of high-end javascript and other “rich” internet applications is going to do to our view of end-user software.

He doesn’t even mention the rise of EC2 and the Google App Engine as sea-changes in the way we buy computational resources, and I think that’s going to have a huge impact.

In the end my prediction is that the way we develop applications will change more in the next 5 years than it did in the last 5, and it’s time to start getting our heads wrapped around these issues, or we’ll be left behind.

And I think that’s the right thing to do, people are ultimately more important than processes. But, at the same time, there’s a paradox to be thought through here, because focusing your management efforts on people can be counterproductive. If you focus on people it’s easy to blame them for failures, to try to change them, and to loose sight of the processes which got you the wrong people, or which made it difficult or impossible for the right people to do the right thing.

In fact much of the benefit of the Lean/Six Sigma camps comes from the notion that you pretty much always get better results if you always assume it’s a process problem, and try to improve the processes.

The solution to the People over Process paradox is easy enough:

Good managers lead by creating an environment where people are empowered, in other words they lead by focusing on a different kind of processes. Processes which put people in charge, and which encourage learning and self-correction. That might sound hard, but really it’s not as complicated as you think. For, example Toyota has thrived by creating a “metaprocess” which gives every employee power over the day to day processes of their job. These meta-processes which make standards of work clear, and make it the team’s responsibility to relentlessly and ceaselessly continue to improve those standards. Employees are expected to think, and to act on a regular basis to improve the way things are done. And they are ultimately “in control” of the processes which govern their work.

If you focus on the right metaprocess, you won’t get into the kind of “process problems” that the agile manifesto was written to combat. Processes will be owned by the people doing the work, not imposed from above, and they will be adjusted continuously to meet the daily needs of the project.

Processes must serve people. But at the same time people following good processes — and even more importantly good process improvement processes — are more productive than people with no process. So, removing process isn’t the answer to “People over process” it’s providing people with control over the processes, and with a well defined way to improve those processes.

If you’re interested, or want to help out, drop me a comment or e-mail me at mark at compound thinking.com

Unfortunately, the students who get into fancy schools like MIT and Stanford and are evaluated both before and after they arrive largely on their individual performance: BUT then life plays a cruel trick on them, forcing them to work in groups, to deal with the messiness and sometimes craziness of human groups — and their individual brilliance is no longer enough and they have all those damn people, with different needs, opinions, priorities, and skills, and different schedules too, to deal with.

If you’ve been selected and groomed based on one set of standards (individual performance) it’s hard to accept that those old standards don’t work any more and your success will be based on a new standard: the ability to work with a group to create, innovate and ultimately produce results as a team.

Individual performance is based on a different set of skills, talents, and motivations than group collaboration. So even if you can make the internal switch to accept the new standards, you aren’t necessarily going to have the right kind of motivation, the right skills, or the right talents to succeed at collaborative work.

Management isn’t easy, it’s hard work that requires really understanding people, their individual strengths and weaknesses, and creatively organizing them to accomplish things as a group that none of them could accomplish on their own. Unfortunately that’s just not what they teach in school, so you have to learn it somewhere else.

Looks like Python and Ruby are both going to have a good number of talks. I’ll be talking about SQLAlchemy, which is the best object relational mapper I’ve ever seen. There’ll be talks about Test Driven Development in Python, Enterprise Architectural Patterns for Python developers, along with lots of cool talks about Lean Software Development, the side benefits of Test Driven development.

You can still submit a talk proposal before November 30th, and you’ll get free room and board. I think it would be great to see somebody talk about Dabo and Desktop application Development in Python, and they seem to be missing any talk about OSX/Cocoa stuff, which I’m sure is because they haven’t had any proposals yet.

It would also be nice to see a good cross platform development with Mono talk…

I’m really excited by the opportunity to get developers of all kinds together and talk about how to be productive and learn more about the strengths and weaknesses of the various tools/frameworks people are using.

But I know that the image of the pointy haired boss is pretty thoroughly ingrained in our culture, and popular mythology exists to explain a shared experience.

Clearly there are a lot of bad managers with MBA’s from prestigious institutions out there.

Henry Mintzberg does more to explain how this happened — and what we might be able to do about it — than anybody else I’ve read recently.

The central argument of Managers not MBAs is that Management as taught by MBA programs is a failure, because they:

• choose the wrong people,
• educate them in the wrong way
• and produce the wrong results.

In other words, the selection process and educational format of most MBA programs actively undermines the practice of good management. MBA’s are a self-selected group of people who “want to get ahead in business,” and the entire program teaches them to compete rather than cooperate.

The result is that graduates of MBA programs have a pretty dismal record at actually starting, expanding, or maintaining stable, productive, businesses.

Mintzberg doesn’t blame MBA holders — they were taught strategy, and accounting, and analysis, but never management.

In particular, MBA programs never taught them the meaning of good management, or the skills they would need to actually grow and manage teams of real people. They aren’t even pointed in the right direction, the structure of MBA programs reward the kind of people who like to compete. And years of “case study” exercises have made them into the kind of people who make snap decisions based on limited data.

MBA graduates generally aren’t the kind of people dedicated to helping other people achieve greatness.

Instead, they want to achieve greatness on their own — which can be a worthy goal. It’s just a terrible goal for a manager. Good managers are relentlessly focused on helping the people they work for perform at their best.

I don’t believe an MBA degree is just a liability.

The MBA’s I know have learned useful terminology, analytical skills, and the training they recieved in in economics, accounting, and business law can be a huge help in the right moment.

An MBA can prove to be an net asset to a manager, but only if you unlearn some skills they teach you (snap judgments, and me-first competitiveness) and make it a point to learn the “soft” skills that are infinitely more important.

The second half of Mintzberg’s book is his proposal for creating better educational institutions, which can identify the right candidates, train them with the right skills in the right way, and ultimately produce much better results than current MBA programs. The broad outline of that plan seems right. But most managers need something they can use to get better at their jobs right now, without spending a lot of money.

• Never stop learning from the people on your team – they are on the front lines learning every day, if you’re not working to learn from them, your loosing potentially critical information.
• Always believe that there is more to their job than making money. Greatness comes from productive passion, and greed doesn’t inspire passion or loyalty.
• Elevate the people who actually produce value — These people actually do the work, and they know infinitely more about the nature of that work than MBA trained managers, so they should be respected, valued, and included (or put in charge) of improvement processes.

Good managers care about people, not head-count, and they strive to make their job a meaningful contribution to a shared vision.

Hopefully I’ll have time to write more about good management tomorrow.

If your future looks bright, make sure your team knows about it. Tell the story of how you achieved all this, thank the people who helped you get there.

If on the other hand your company’s immediate future is bleak you can still encourage vision, but you have to creatively and courageously tell stories. I’ve been through this kind of thing at work, and we did come out stronger.

Unfortunately I can’t tell that story.  Fortunately I thought of an even better story to articulate this idea of vision.

In a country where segregation was the law of the land, and racial hate was rampant Martin Luther King Jr, continued to tell the story of a future where people would no longer be judged by the color of their skin.  He continued to tell these stories in the face of economic hardship, and in under the the constant threat of assassination, and with the memory of attacks on his family still burned deeply into his memory.  It is context that  Martin Luther King Jr. made one of his most powerful speeches to his battle weary friends and followers:

Well, I don’t know what will happen now. We’ve got some difficult days ahead.

But it doesn’t matter with me now. Because I’ve been to the mountaintop…

Like anybody, I would like to live a long life. Longevity has its place. But I’m not concerned about that now. I just want to do God’s will. And He’s allowed me to go up to the mountain. And I’ve looked over. And I’ve seen the promised land. I may not get there with you. But I want you to know tonight, that we, as a people, will get to the promised land.

If you are faced with a Death March style project, the only thing you can do is to hold out a vision for a better future and do everything in your power to make that happen. You can’t ignore the present, you can’t ignore the grueling overtime, the unrealistic deadlines, or the managerial mistakes that got you here — if you do nobody will listen to you, nobody will respect what you have to say.  Instead, you have to demonstrate that you have or can come up with a plan to get out of the desert into the promised land.   Compelling vision begins with the hard truth of today, but holds up an achievable hope for tomorrow.

Premature automation is also the root of even more evil

If you spend time and money to optimize the performance of a particular piece of code, it will often become more complex, more difficult to understand, and more costly to change in the future.

The same thing happens when you automate a sales process, or the way you route phone calls. The process becomes less adaptable to change, sub-processes are invented to route around difficulties with the main automated process, and can get crazy.
Sure, you can get things done more quickly. But if you’re customers are routed incorrectly, they don’t care how quickly they get to talk to the wrong person!

And the danger is not just that you’ll get it wrong in the first place, but that you won’t be able to adapt to changes in the environment around you because adaptation would mean throwing out all the work that went into optimizing that process.

Some of this problem is economic.

The cost of automating a process must be repaid quickly for the automation work to be valuable. The exact ROI time is function of how much change happens in that area of your business.

But some of it is psycho-social.

Even if a particular automation project has already paid for itself, the fact that it exists makes the whole organization more likely to resist change, because you don’t want to ‘throw away’ all the hard work that went into automating the old process.

Not only that, if it’s not handled well by management, seeing a project you worked on for months last year being “thrown away” can be seriously demotivating to the automator.

Premature automation is the root of all kinds of evil.

I’ve decided that it’s time to change jobs. I just finished the TurboGears Book, and several Python related articles, and all of that reminded me how much I love Python, Agile Development, and teaching people.

Unfortunately, I don’t get to do any of those things enough in my current job.

Don’t get me wrong Humantech is a great company, that helps prevent a lot of unnecessary pain and injury, and I get to work with some fantastic people. So, it pains me to leave, but I don’t get to focus on doing what I do best. Instead I am pulled in a million different directions, and I find I spend more time than I’d like on things like monitoring people’s cell phone bills.

My main goal in making this change is to find something that lets me enhance my software development skills, or turns me loose in the coaching/training/advocacy side of things — preferably something that lets me do both.

In particular, I’d love to work on an XP team that’s actually walking the walk, and not just talking about how cool Agile is. I’ve been a fan of the ideas behind the whole Agile movement, and I’ve implemented some at Humantech, but it would be great to work with people who’ve been there, done that.

I also love talking to developers (and managers) about stuff that could improve their lives. I love training, and I think one-on-one coaching is an amazing opportunity, and would be thrilled to have a chance to do more of that as part of my regular job.

I’m looking for something in the Michigan area (preferably Ann Arbor) but I’d definitely telecommute, and travel for the right job.

If you, or somebody you know is looking to hire someone, let me know (Mark.Ramm at gmail dot com), and I’d be more than happy to talk to you.

In the article I talk a bit about a project I’ve been thinking about for several weeks, which uses TurboGears to create a ticket tracker based on my ideas of how to use David Allen’s Getting Things Done system in a group setting.

It’s called Action Flow, and the idea behind it is to make it easy to capture all of the next actions for all of your projects in a simple system outside of your head. The difference between this and other similar systems is that it is designed to track actions and projects for individuals and groups in the same free-flowing self-organizing way that Agile Software projects are run.

I personally want to use it to run an IT department, and track all the various tasks and projects we have going. It’s not there yet, but as I finish up the final edits on the book I should have more time to work on this project.

I’m also hoping that it will be a good place for new folks to Open Source and TurboGears to get involved on the ground floor of a project that’s very welcoming to anybody interested, even if you don’t have experience. Learning experiences are generally effective to the extent that they require active participation from everybody involved, so I think this will a great way to learn more about web development and TurboGears.

So, if you’re new to TurboGears, new to Python, or just want to help, feel free to join the mailing list and introduce yourself. We’ll be nice, and help you get involved and get learning TurboGears the fastest way possible, by working with other people on a real project!