Sometimes we think we know that a piece of code is important so we spend some time optimizing it. And in the end it’s less clear, and less maintainable, and it turns out that our bottlenecks are all elsewhere.

But, sometimes we do know where bottlenecks are going to be, we’ve learned from experience, and we know what needs to be done.

We know that architecture determines performance, and architecture isn’t easily bolted on at the end of the project.

So we have a conundrum. We shouldn’t optimize yet because we don’t know where the bottlenecks will be. We shouldn’t wait to optimize because we can’t easily retrofit a good architecture on a complex system.

Some of the conundrum is only apparent — there’s a difference between architectural problems that need to be set up front, and the kind of low level micro-optimization that obscures more than it helps. But, sometimes these conflicts are real — how do I know if I need a multi-process multi-consumer queue system for PDF generation before we build the system and benchmark it? If you don’t need it, that kind of extra architectural complexity just obscures the bit of code that actually solves the problem.

Solving the problem by going meta

Perhaps the problem really is that we’re dumb and optimize the wrong things at the wrong time. The solution to that problem is to get less dumb. Which means that we ought to spend time optimizing “learning”, both within our project processes, and across projects.

Codifying this learning is what the Patterns of Enterprise Application Architecture book was all about.

And I think it’s great as far as it goes, and if you haven’t read it you should buy it now.

But there are a lot of patterns that I can identify from my last half dozen projects that aren’t covered in PoEAA, so it would be great to see a next generation of books and blog posts that cover the modern architectural trade-offs that you have to make, something that covers some of the paterns of the web.

Scalability via in HTTP, etags, caching, and load balancing (the whole RESTful services argument), networked async processing patterns, etc. Scaling to the public web levels requires a whole different set of architectural principles than scaling to the old “enterprise” levels did, and that knowledge seems very much in flux.

It would be great if it also provided some advice for those of us who’ve moved into what Neil Ford has called the world of the Polyglot Programmer, patterns for coordinating activities across language barriers in a sensible way. That’s part of the nature of modern web systems too.

I haven’t gotten to know many people outside the engineering team in Dexter, but they are great guys.

There’s lots of good stuff happening here, support for bazar, mercurial, git, trac, and other options on SourceForge itself, improved feeds, and other API’s for getting at SF data, etc. But I’m only peripherally aware of all that at the moment because I was hired to work on “totally new stuff” which is written in Python.

What I’m working on

Our first new project is a site called FossFor.Us, and it was the vision for this site, and the team that is working on this and other new stuff, that sold me on the coming to work for Sourceforge. It’s written in Django, and it’s been my first really large Django project, and while the experience has been pretty positive, there have been a number of things that have renewed my commitment to TurboGears development — but that’s a blog post for another day.

The backstory to the FossFor.Us site is that open source project hosting providers (Sourceforge and it’s recent competitors) have traditionally been pulled in two very different directions by two very different sets of users:

• developers of open source software
• and people who just want to.

And that tension has held us back in the past, we have to serve everybody with the same portal, and it ends up not serving either community as well as it should. But since developers are the most vocal users, it’s been the second class of user that’s been most neglected.

These people are just looking to get things done, and don’t care about the “project” part of open source software, they are, at least at first, only interested in the “product.” In many ways the Free and Open Source Software community has not served these people well.

Fossfor.us is in it’s first incarnation an attempt to create a window on the free software world, that’s just about finding and using software. But in a larger sense it’s an attempt to help us as a community to connect with potential users better.

I think connecting FOSS geeks and users is actually important

It’s important because people aren’t aware that there are free options, and are paying for software they can’t afford. There’s a prototypical user (based on a real person) that we talk about a lot, who’s a single mom, has an old laptop, and struggles week to week to pay her bills, but who bought Photoshop, because “that’s how you edit photos.” Her family could have used that money to more productive ends, but because she needed to edit photos, and didn’t know about the free alternatives all those opportunities are just lost.

Of course the same thing is true of small business owners, who could use free software to reduce their “overhead” costs, and actually spend money on creating things people love. Free software has the potential to lubricate the wheels of the economy, encourage entrepreneurial activity, and enrich people’s lives.

All of this is to say I think fossfor.us is a way to serve the world by making the product of all the open source developer’s labor more easily available and more accessible to real people. And when my mom actually used it to find some software a couple weeks ago, I knew we’d done something right.

Jacob rails against those who say the differences between modern computer languages amount to nothing more than “syntactic sugar.” Ultimately he argues that: Syntactic sugar matters.

Sure it makes no “technical” difference but it does make a huge difference — because it changes the way we think about writing software.

I’ll loop back to that in a second, but first a quick detour through another recent blog post:

Eventually you come to realize that in order to truly succeed, you have to write programs that can be understood by both the computer and your fellow programmers.

Of all the cruel tricks in software engineering, this has to be the cruelest…. Even when you’re writing code explicitly intended for the machine, you’re still writing. For other people. Fallible, flawed, distracted human beings just like you. And that’s the truly difficult part.

–Jeff Attwod

The thread that ties both of these together is that they highlight the way that people and by people I mean software developers, tend to forget is that code is always two things:

• A series of instructions or declarations processed by a computer.
• A series of instructions or declarations processed by one or more human beings.

Code is a machine construct, but it’s also a social construct. Software engineering is a strange name for our discipline, since the hard work of programming isn’t just getting code that machines can run, but in creating abstractions that allow human beings to learn, understand, and evolve the code over time. We didn’t invent Structured Programming, or Object Oriented Programming because they help the computer understand what we mean — we created them because they provide us with tools to help us as human beings to be able to understand the code we write.

Non-software Engineers aren’t concerned primarily with the practice of communicating complex thoughts and ideas to others. This is a vast oversimplification, but I think it’s fair to say that they are interested in constructing mathematical models of how things things behave, so that they can build stuff that works.

But that’s not what we are, we are creative writers, we invent new ways of thinking about the world, and we try to communicate them to each other every day.

Software Engineers do create incredibly complex systems, and operate under constraints that other writers do not — what we write has two audiences, one human, and one non-human. Writing for the non-human audience alone will result in code that’s incomprehensible to other programmers, but the opposite is also true. Computer programming is hard — and it’s hard for a very specific reason — because it requires thinking like other people, and not thinking like people at all.

Fortunately, the problem is simplified a bit by the fact that other programmers have learned at least somewhat to think like silicon and metal, so you can lean a little bit in that direction and still be understood. But still you have a very exacting, very alien audience, and a very exacting, very human one — and programming languages must be designed to balance the needs of both.

So, perhaps we ought to stop calling ourselves software developers, software architects, or software engineers, and start calling ourselves software writers.

REST is software design on the scale of decades: every detail is intended to promote software longevity and independent evolution. Many of the constraints are directly opposed to short-term efficiency.

Unfortunately multi-process and the JVM are currently two tastes that don’t taste great together. You can do it, but it’s not the kind of thing you want to do too much. So, the Jruby guys had a problem — Rail’s scalability story is only multi-process (rails core is NOT thread safe), and Java’s not so good that that….

Solution: Running “multiple isolated execution environments” in a single java process.

I think that’s a neat hack. The JRuby team is to be congratulated in making this work. It lets Rails mix multi-process concurrency with multi-threaded concurrency, if only on the JVM. But it’s likely to incur some memory bloat, so it’s probably not as good as it would be if Rails itself were to become threadsafe.

I’m not sure that the Jython folks have done anything like this. And I’m not sure they should. It’s a solution python folks don’t really have. Django used to have some thread-safety issues, but those have been worked out on some level. While the Django people aren’t promising anything about thread safety, it seems that there are enough people using it in a multi-threaded environment to notice if anything’s not working right.

At the same time, TurboGears has been threadsafe, from the beginning, as has Pylons, Zope, and many other python web dev tools. The point is, you have good web-framework options, without resorting to multiple python environments in one JVM.

Why you actually want multi-threaded execution…

In TurboGears we’ve found that the combination of both multi-threaded and multi-process concurrency works significantly better than either one would alone. This allows us to use threads to maximize the throughput of one process up to the point where python’s interpreter lock becomes the bottleneck, and use multi-processing to scale beyond that point, and to provide additional system redundancy.

A multi threaded system is particularly important for people who use Windows, which makes multi-process computing much more memory intensive than it needs to be. As my Grandma always said Windows “can’t fork worth a damn.” ;)

But, given how hard multi-threaded computing can be to get right TurboGears and related projects work hard to keep our threads isolated and not manipulate any shared resources across threads. So, really it’s kinda like shared-memory optimized micro-processes running inside larger OS level processes, and that makes multi-threaded applications a lot more reasonable to wrap your brain around. Once you start down the path of lock managment the non-deterministic character of the system can quickly overwhelm your brain.

As far as i can see, the same would be true for a Ruby web server in Ruby 1.9, where there is both OS level thread support and an interpreter lock.

I’m well aware of the fact that stackless, twisted, and Nginx have proved that there are other (asynchronous) methods that can easily outperform the multi-threaded+multi-process model throughput/concurrency per unit of server hardware. The async model requires thinking about the problem space pretty differently, so it’s not a drop in replacement, but for some problems async is definitely the way to go.

Anyway, hats off to the Jruby team, and here’s hoping that Rails itself becomes threadsafe at some point in the future.

What is Ruby 1.9? Do we have a spec? A test suite? Anything? If the answer is no, it’s not ready.

–Charels Nutter (in a comment here)

Partly that’s because Charels wants 1.9 as a reference implementation — not just usable interpreter.

But, that highlights one of the reasons I like the way python is developed. There is a test suite, there is a reasnably complete set of specifications. So, from here, it looks like Python 3 is a lot “more done” than Ruby 1.9. But that’s OK, they are both way more done that Perl 6. ;)

I’m still looking forward to Ruby 1.9, and python 3.0.

Python get’s a very positive review:

Today you’d be nuts not to look seriously at PHP, Python, and Ruby.

So, the rise of the so-called scripting languages is one of the inflection points, but it’s not the only one.

He singles out web-framework development as one place where there’s a lot of stuff happening, and a lot of new “rails-like” frameworks are cropping up all the time. TurboGears will live or die in the context of a much larger web-development revolution, and we need to be prepared to make our way forward in the midst of that.

What comes after rails will not be a rails clone. It will learn the right lessons from rails, avoid the pitfalls of rails, but it will also need to carve out something new and better than rails. For RDBMS users, I think the key difference between TG and Rails is the power and flexibility of SQLAlchemy. We need to “sell” this better.

There are a lot of other revolutions coming according to Tim. And I do think we’re looking at big changes in terms of everything from programming language choice, to web-development tools, to end-user desktops, and data persistence mechanisms. We’re also just beginning to see what the world of high-end javascript and other “rich” internet applications is going to do to our view of end-user software.

He doesn’t even mention the rise of EC2 and the Google App Engine as sea-changes in the way we buy computational resources, and I think that’s going to have a huge impact.

In the end my prediction is that the way we develop applications will change more in the next 5 years than it did in the last 5, and it’s time to start getting our heads wrapped around these issues, or we’ll be left behind.

But he does, finally, at the end of his article, he get around to a quick mention of Jython in the context of asking a bigger question:

Will either matter to Java? After all Jython’s been around for a long time without making a huge impact on the JVM. Tool support is frankly pathetic for any of these languages when you compare it to what you have for Java at the moment.

This question is skating right around the critical issue, without ever touching it directly.

Why is there so little uptake for dynamic languages on the JVM? And what will it take to change that?

Another way to ask the same question would be: We have a lot of deep experience with C integration in the Perl/Python/Ruby communities, but there is no such deep experience with the JVS as a platform for scripting languages. Why?

To sharpen the question a bit further, we should note that the Jython/java integration story is in many waysbetter than the Python/C story. In Python 2.5, c-types makes using C libraries from python easier than ever — but you don’t have to do anything special at all to use Java libraries from jython. In spite of this, Jython and all the other new “dynamic languages” on the JVM have not become a popular way to work with Java libraries.

Why?

I’m going to address this from the perspective of Jython user, because that’s the JVM/dynamic language experience that I have. But, I’ve played with Jruby enough to know that the issues are similar.

One problem that seems to stop python programmers from flowing over into Jython easily is the impedance mismatch between the way things are done in Python and the way they would be done in Java. As a python developer you find yourself constantly switching mental gears to write anything complex in Jython. You’re thinking about the python code you’d write, and things sort of flow until you run up to something in a java library which requires you to look at the world in a fundamentally different way.

Of course, the same is could be said to be true of Python and C. Sure, you could argue that C programmers and python have more similar pragmatic approach to library development, but I don’t think that’s the main difference. The main difference is that developers tend to “wrap” C libraries and provide “pythonic” bindings which reduce the impedance mismatch. Nobody seems to do this in jython, partly because it’s just too easy to ignore the need for good pythonic API’s and just get something done. It seems that easy is sometimes the enemy of good — which is definitely not news to any experienced programmer.

But, perhaps there is another possible reason, is that the JVM was written explicitly for java, and is not particularly friendly to dynamic languages. Reflection is slow, and in general features used by dynamic languages haven’t been a priority for the JVM. Fortunately there’s some work happening already to make the JVM a much, much friendlier place for dynamic languages. But I don’t expect that Jython is going to be faster than C Python any time soon. So that, means that the JVM as a platform needs to offer something more than just a good way to implement python. I’ve heard some rumblings from the Jython team that this may not be a significant factor in the future, but it’s certainly colored people’s perceptions in the past.

This brings us back to the fundamental nature of the JVM as a viable platform. It’s one thing to have languages that run on the JVM, and call that the platform, but the language and the managed runtime environment are only part of the picture — in my experience the platform is defined as much by it’s libraries as anything else.

I think this is the main problem that the JVM will face as it evolves into a multi-language platform over the next few years — the java standard library is complex in unexpected ways (how do you open a file and process it line by line — I always have to look it up?) and that makes it difficult to use as the foundation for a vibrant scripting platform.

If however people aren’t working with java libraries, you’re just running Jython on the JVM, and all you get is a slower slightly more out of date version of Python, with fewer libraries, and that’s not a very compelling case. Perhaps you need to run on the JVM for political reasons, or you need access to one of the java libraries that don’t have an obvious python equivalent, like i-text or lucene. But except for those relatively narrow use cases, why wouldn’t you stick with CPython?

The Microsoft threat

Microsoft intentionally has worked to create libraries that make sense for C# users, but which are accessible from languages like VB.net, and IronPython. Thus they have created an alternative Python platform which is much more interesting than Jython. And to be successful in turning the JVM into a multi-language platform I think something similar needs to happen to Java.

At least right now, the situation the way that Groovy, Jython, Scala and JRuby have been implemented makes it unclear how well you’ll be able to integrate libraries written in one of these languages with libraries from another. Effectively this places Java libraries at the center, and relegates dynamic languages to “second class” status. At this moment, writing libraries in Jython that would be in an attempt to make them usable to Jruby and Groovy folks seems like a fools errand.

So, having a common core of reusable libraries written in Java seems like a key ingredient in making the JVM a better platform. These libraries would have to be built:

* with multi-language reuse in mind
* in ways that don’t require lots of boilerplate code
* to work well with dynamically typed languages

That kind of standard library would benefit allof these new players on the JVM. And that’s what I think Sun ought to invest in if they want the JVM to become a platform that JRuby, Jython, Scala, Groovy, and whatever else comes next, can thrive.

Another Microsoft Threat

At the same time, Microsoft has been widening the gap in another way, by introducing the DLR — a new set of tools for dynamic language implementors.

Much of the thinking behind the DLR comes from Jim Hugunin (the original implementor of Jython), and if it works the DLR will help to make dynamic languages first class citizens on the Microsoft CLR since you will be able to use them to write libraries that are cross-compatible, making IronPython libraries usable in IronRuby, etc.

The aforementioned project to make the JVM more friendly to dynamic languages is a good place for language implementors to work together. And it would be very interesting to see the jython, jruby, and other language implementors get together and provide a good solid answer to the DLR. I’ve been talking to some of the Jython guys this week, and they tell me that this is happening, and that a lot of the work is already done.

Given the politics of language communities, and the necessities of the Open Source development model, I’m sure this has been more difficult than it could have been, so I’m very excited to hear that the people involved have taken the time to make that happen. I haven’t seen a lot of talk about this on the internets, but it looks like people are quietly doing the right thing without running the hype machine in overdrive which is refreshing.

After talking to the Jython guys this week, I’m actually pretty optimistic about the JVM. It’s growing in a direction that provides a solid memory-managed platform for the dynamic languages of the present and future, and it could become a platform for collaboration between language developers.

The CLR could do the same thing, but I want to see it happen on the JVM, because that would make it free in all the important senses of the word.

I’ve created a page on the Wiki for Sprint Organization tasks: http://docs.turbogears.org/SprintOrganization

If the sprint goes well it could get us very, very close to the point where we could reasonably do a TurboGears 2 technology preview release.

The main things we’ll need to do is sync up with the latest pylons, improve our tests, and do some basic doc organization work (migrate some information from the doc strings to the docs wiki, and create some pages which link to external docs). There’s lots of other things I want to do like improve out user authentication/authorization/registration support, or create a toolbox tool for helping create SQLAlchemy models. So, there’s tasks for anybody who can lend a hand.

If you’re available and want to help out (either attending physically or virtually), feel free to add your name to the wiki.

Likewise, if you can host a in-person sprint in another location, please feel free to edit the wiki with that information.

I loved the part about the database as an “implementation detail.” That sings to me.

–Daniel Waite

And, in spite of the poetic hyperbole of folks like Daniel, I think relational databases exist for good reasons and knowing about how they work is a good thing. Active record treats the database kinda like a big hash table in the sky, which is OK for some applications, but not so OK for others.

I find that large, complex apps have a way of growing in unexpected ways over time — and relational algebra is much better designed to handle unexpected requests for complex datasets than hash tables or even the most well designed objects. SQL, and relational algebra are implementation details in the same way that using a hammer is an implementation detail in building a tree house. Sure you can do without, but you will have a harder time, and will definitely end up building a different treehouse!

Don’t misunderstand me, Object’s are good things, and I sure as hell hope people understand them, use them, and make great software with them. But relational algebra is a good thing too. And it’s become clear that any really good database toolkit can’t forsake one in favor of the other, and that’s why SQLAlchemy, Storm, DejaVu, and Hibernate are all better tools than Rails’ Active Record.

As for Derik’s original article. In spite of all the brouhaha, Derek’s post basically says: “hey I liked Rails, and I learned a lot about good programming from it. And ultimately it was easier to get my site rewrite done in PHP because I learned rails first.”

I think it’s obvious that Rails could have done the job for Derek, but two things got in the way:

• everything was already in PHP so Rails wasn’t and incremental update, it was a rewrite.
• Active Record got in the way of using SQL as a relational algebra engine, rather than a glorified hash persistence mechanism.

TurboGears users should pay attention to both points. If you have a large complex app, you’ll be better off with an ORM that treats SQL as a full partner, which is why we’re migrating the defaults to SQLALchemy which does a fantastic job of making both relatinal algebra, and object oriented programming part of it’s core.

The second point is much harder to deal with, but if you can wrap existing application functionality in a larger TurboGears app, and migrate it piece by piece you’ll definitely have more success. I’ve been doing a lot of Web Service architecture stuff this year, and that’s proven to be a great way to connect existing resources to TurboGears applications.

At that conference one Ruby/Rails book author said something like; “I really like the vibe here. Everybody here is talking about making a difference, educating third world children, building better communities, and making the world a better place.” And he was setting this as a stark contrast to his experience of RailsConf where “everybody was talking about how much money they made.”

On one level it’s hard to fault the Rails conf people for this. Money does on some level validate what they’ve been doing, and it isgood to feel that kind of validation.

But, I think it’s fair to say that the Rails people have been too focused on proving something, and that has lead to a perception that many rails folks have a chip on their shoulder, or are totally focused on money as a metric of success. The extent that some elements in the Rails community have embraced this perception (as mentioned on Martin Fowler’s recent blog posting), is actually a bit frighting.

So, I think Chad is doing exactly the right thing by encouraging the Rails community to take a look at how they can improve their image. A couple days ago Chad posted a blog entry entitled “Change the world” and I hope that it makes an impact in the Rails community. I definitely see Chad Fowler many of the other members of the Rails community that I know personally as good guys, and I’d like to see the current negative stereotyping of the rails community come to an end.

I’d like to see the Rails community as a whole step up to the plate, and focus their attention and marketing efforts on new projects with altruistic goals. Turn the whold Buzz machine towards something that will really change people’s lives — replacing Java just isn’t a worthwhile goal when compared with helping to feed hungry people.

So, in the hope that a little good natured competition can spur things forward, I’d just like to mention that I think the Python community is beating the pants off of the rails community in the change the world department. Here are a couple of projects that I could think of off of the top of my head which the Python community is working on:

• Irrepressible.info — which is focused on fighting censorship
• the OLPC project — which aims to bring better educational materials to millions of underprivileged childeren around the world
• the Open Planning Project – which aims to provide tools for grass roots community improvement tools.

On another note, over the last 6 months or so Compound Thinking has been been working for Philantech on a project called PhilanTrack which takes a slightly different tack. The goal of the Philantrack project is to make it easier to manage the process of giving, receiving and reporting on grants.

The idea is that “greasing the wheels” of grant giving, and allowing non-profit organizations to focus more on doing the work, will make the nonprofits more efficient, more fun, and ultimately make the world a better place.

We may not think about it, but we geeks have a lot of individual power, and collectively we really can make the world a better place. If you’re working on a Rails or Python app that can change the world for the better, drop a comment here and let us all know.

If anybody is willing to show up in Ann Arbor, I can provide food, a place to hack, and some good clean post-sprint debauchery. For, unfortunately those sprinting outside of the Ann Arbor area will have to provide their own space, food, and most of all their own debauchery.

If you’re interested in participating in the sprint, please join the Sprint Coordination mailing list.

We’ve got folks committed to:

• Migrating test infrastructure to CherryPy 3
• Finalizing the TurboGears 1.1 configuration system, and implementing what we decide on
• Workiing on the turbogears SQLAlchemy integration to make it easier to use multiple databases
• Doing some general clean up of open tickets

My personal goal is to help get the trunk into a usable state, which means tackling the configuration issue. Once people can run their existing projects on the trunk, TG developers (like me) will be able to do our daily work on turbogears projects on the trunk. Which in turn should help us to attract more developers and testers and continue to make TurboGears a great tool for rapid web application development.

Looks like Python and Ruby are both going to have a good number of talks. I’ll be talking about SQLAlchemy, which is the best object relational mapper I’ve ever seen. There’ll be talks about Test Driven Development in Python, Enterprise Architectural Patterns for Python developers, along with lots of cool talks about Lean Software Development, the side benefits of Test Driven development.

You can still submit a talk proposal before November 30th, and you’ll get free room and board. I think it would be great to see somebody talk about Dabo and Desktop application Development in Python, and they seem to be missing any talk about OSX/Cocoa stuff, which I’m sure is because they haven’t had any proposals yet.

It would also be nice to see a good cross platform development with Mono talk…

I’m really excited by the opportunity to get developers of all kinds together and talk about how to be productive and learn more about the strengths and weaknesses of the various tools/frameworks people are using.

Yesterday Last Week, I in response to a post by Dave Thomas, I wrote about optimizing for readability.

Dave Thomas post basically amounts to an homage to Ruby at the expense of Perl.

Dave likes Ruby because it is far more readable, and therefor more maintainable than Perl. So far I’m totally on the same page. Readability is critical to maintainability and extensibility, and that’s where most of the work on any successful software project is going to be done.

But, here’s where we part company at least a little bit.

I happen to prefer Python over Ruby because optimizing for readability is much more a part of the philosophy behind Python. Most of the lines from the Zen of Python have readability implications:

Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.

Ruby makes better choices than Perl

But the language is still built around something they call “the principle of least surprise,” and the least surprised person here is the initial developer, not the code reader.

In practice means that there are several different names for the same array methods, and many array methods that are only slightly different from one another. If you are familiar with another language’s way to do it, the “least surprising” thing would be for that same method to just work on Ruby arrays right?

And this can make it easier to get started with Ruby. But there’s a downside, when reading Ruby code this means you may need to learn and remember all 78 different array methods. Some of them are obvious, .last will give the last element in the array. No worries there, and I like humane interfaces which give you easily understood and convenient functions, but this is just one example of how Ruby takes it too far, by forgetting that you will eventually have to figure out what eacy of those functions does in somebody else’s code.

Quick Ruby Quiz:

• is there a difference between array.size and array.length ? (no)
• How about array.map! and array.collect! ? (no)
• What about array.delete_if and array.reject ? (yes, reject returns nul if no changes are made)
• And for extra credit, why is there an array.collect (non-destructive) and an array.collect! (destructive) but not an non-destructive version of array.map! ?

I’ve been reading more Rails code recently, and while I like Ruby, and I admire Rails, I find myself constantly wishing I could hold more of this stuff in my head. I learned to write Ruby pretty quickly, but reading other people’s code has taken longer — just because there are lots of nooks and crannies in the built-in classes and modules that will take me a while to learn.

But Python makes better choices than Ruby:

Don’t get me wrong, I like Ruby. And it’s not particularly difficult to read. But the philosophy of the language designers led to design choices that emphasize writability over readability. And in that department I think the advantage has to go to Python. Python lists are easy to use, but more importantly I understood all of the list methods and how to use them in the matter of a few min. Perhaps Ruby’s arrays are more powerful that Python lists, but so far I’ve yet to find something that can be done in Ruby that can’t be done easily in Python.

I mean, think about it — even the things people complain about in Python, like the explicit self or significant whitespace, are designed to with readability in mind.

But I think there is one optimization that is never premature — readability.

I’m not the only one who sees readability as a key metric in writing maintainable software. Dave Thomas (author of The Pragmatic Programmer, and Ruby advocate) recently wrote the following:

But there was a problem [with my use of Perl]. Perl is a great language…. But even on my most charitable days I could never really claim that these Perl programs were exemplars of readability. And, in turn, this lack of readability made it hard for me to pick one of them up six months after I’d written it and make some change—even small alterations could involve long periods of head-scratching as I tried to work out exactly what the entries in my hash of arrays of hash references actually contained.

Amen, Dave! If you can’t read you code: you can’t adapt it to changing requirements, you can’t build new features into it, you can’t optimize it’s performance. In other words, you can’t maintain it.
And the documented truth of software development is that most of the time and money spent on any sucessful software product is going to be spent in maintenance and upgrades — not initial development.

Writing code that is difficult to read is like running up your credit card buying expensive toys; even though you don’t pay today, the bill is going to come due eventually — and the longer you wait the more it’ll cost.