I think the main thing I’m talking about is the ability to map objects to relations, not just tables. So in SQLALchemy (and many other ORM’s) you could for example, do a join across three tables, and map an object to that result set.

In active record you can use the results of a custom SQ L query, but you can’t modify and save back the results.

There are other features that I expect in a “full featured” ORM, like having an “identity map,” following the “unit of work” pattern, playing nice with database features like multicolumn primary keys, foreign key constraints, etc.

I know I just threw out a couple of terms you may be unfamiliar with, if so, there’s a great book — Patterns of Enterprise Application Archetecture — by Martin Fowler, which explains a lot of the basic principles of database access. And it’s that book which coined the terms “identity map,” “unit of work,” and even “active record.” I highly recommend it.

But it is possible that I don’t get your meaning. I don’t really know the difference between what would be considered simple vs. full featured ORMs.

* Users who want to do everything by hand can just use SQLAlchemy’s connection pool and configuration, and issue straight hand-written SQL against the connection. This offers a much cleaner and easier to work with option than the straight DB-API.

* People who love SQL but want to be database independent, and want to avoid object-relational mapping can use SQLAlchemy’s table objects and SQL generation API.

* There will of course be people who are interested in using the power of the SQLAlchemy ORM, who can leverage the great mapping capabilities of SQLAlchemy following the data mapper pattern for maximum flexibility.

* People who want something a little bit simpler than SQLAlchemy’s ORM and who prefer the active record pattern can utilize Elixir with minimal switches flipped in their project, giving them a cleaner API, and they won’t sacrifice the great ability to drop down to SQLAlchemy’s table objects or custom mappers when needed.

Its the best of all worlds, and will make TurboGears be able to not make the database layer just an implementation detail, but allow users to harness the raw power of their relational database of choice.

Keep it up, Mark!

Any way, when you need ad-hoc queries use a relational database. If you need a relational database while programming with objects and you can live with the performance penalties you get with using an ORM go ahead and use an ORM. They will make your life easier.

Note that most applications don’t need to worry about these performance issues so it is wiser to just use the ORMs so as to develop the code faster. Those that are concerned about the performance issues probable shouldn’t use a relation database to begin with. At least this would be my advice for typical web applications.

When you use an ORM it is important to understand how the ORM works under the covers so that you don’t do dumb things with it. If you don’t take the time to understand how they work you can easily build applications that have terrible performance issues and give the ORM a bad name.

Now what about the selection between a simple vs a full featured ORM. I agree with Mark that you shouldn’t use an ORM that tries to hide the database from you. I know it is very tempting for most developers to just use a simple ORM as it takes very little time to get up and running with it.

But sooner or latter you will have performance issues or you will want to do something that the simple ORMs was not designed to handle. You will have to fight with it and will have to learn the inner workings of it to understand why you are running into these performance issues.

If in the end, you’re going to put all that effort to learn how to use a simple ORM you would be better served by taking the time to learn SQLAlchemy. It will not require that much more effort to learn SQLAlchemy than a simple ORM. Although, you will have to put in a larger effort to get up and running with SA but the gains from the additional capabilities will more than make up for this extra effort.

With SA you will not find it necessary to fight with it. At least no where near as often as you would with the simple ORMs. Plus when ever you need to do something that the ORM layer was not designed to handle you have the option to drop to a layer below the ORM which is not possible with the simple ORMs.

What about Elixir (a declarative layer on top of SA)? My advice is not to use it. It will take less time to learn like the simple ORMs but sooner or latter you will have to learn more about SA and very likely the same amount of information as would be required had you just used straight SA’s ORM.

So in the end you will learn more and the only gain will be that you can save yourself a little typing. To me it just doesn’t seam worth the extra effort. Of course if you do use Elixir you can still drop to the lower SA layers as necessary.

You have a good point. There’s plenty of room for non-relational data storage, from plain text files to ZODB.

My point is not that you should always use relational data storage. In fact, that would be absurd! I don’t want to store my python source code in a relational database, I want plain text.

Instead, my point is that if you need the ability to do post-hoc queries you need the “relational” part of relational data storage, and you shouldn’t use an ORM that hides that from you.

If on the other hand you just want to persist some objects to disk can live with a slightly simplified data query syntax, ZODB is going to make your life a lot easier than ActiveRecord would.

To stretch the original analogy a bit, if you’re building with concrete and steel, you may not need a hammer. But I definitely wouldn’t want to build a wooden tree house without one.

- relational vs non-relational databases
- simple vs full featured ORMs

You should just concentrate on the simple vs full featured ORM issue as the choice of database type is a separate issue. After all ORMs map to relational databases.

On the relational vs non-relational databases issue.

There are many types of hammers and choosing the ideal one for a particular job can make a world of difference in getting a task done. Would you use a 10lb sledge hammer to drive a tack into a will? I don’t think so, as you will likely smash your hand holding the tack.

So what kind of hammer is a relational database? Is it a common hammer. What about BerkeleyDB? A nail gun as its faster and not as flexible as a common hammer. What about ZODB…

Seriously, there are many types of data stores and they are all tools. The one who better knows the pros and cons of each tool can make wiser decisions on their use and will likely get the job done more efficiently. So don’t get fooled into believing that relational databases can solve all your problems and in a better way than other tools as in many cases they are the wrong tool to use.

I prefer to “work smarter not harder” so I make sure my tool belt is full of tools.