ORMs and persistence ignorance make perfect sense: if I am programming in a language, then I want to use the language's native implementation of objects and data structures to drive my program. I shouldn't need to worry about SQL or a database, and I always thought that inline SQL strings in a program was very cumbersome.

Why is SQL still used as a backend? Why doesn't the ORM just persist objects to XML or a binary file(s) and avoid all the overheads and difficulties of administering a database server?

  • 2
    Have you looked at NoSQL databases? If not, I really think it's a field that would interest you. – Carson63000 Dec 14 '11 at 4:23
  • @CraigJ Great: all set. Thanks for your patience and understanding. :) Interesting question, by the way. – user8 Dec 14 '11 at 7:48
  • 4
    @CraigJ: You do realize that ORM stands for Object Relation(al) Map(per)? ORM's were designed to bridge the gap between an Object Model and the restrictions of a Relational Model. Persistence ignorance was coined as a term to convey that using an ORM meant that you didn't have to worry about which relational database was used to persist your objects. Persisting to other storage means, like xml files was added to support the briefcase model. ORM's may have outgrown their original design intention, but that doesn't invalidate the usefulness of relational databases. – Marjan Venema Dec 14 '11 at 9:23
  • 2
    Not all databases have a lot of overhead. You might want to learn a little about sqlite, for instance. – Bryan Oakley Aug 9 '12 at 12:47

The answer presented here is a quick answer and is not comprehensive, since the subject is rather big. Your question is about 3 things (at least):

1- Databases

2- SQL

3- ORM Concept

A database is any repository of data. You can store your data wherever is appropriate for your application. SQL is a way to access this data when the data usually, SQL is used to access and manipulate RDBMS. ORM represents a way to access data from within applications using SQL or other means.

My question is: why are we even keeping SQL...

If your database is not a relational database or does not beinfit from what relational database systems, you may not have to use SQL. In the past 30 years relational database have answered so many business needs and it has grown to be robust, efficient and reliable. The conceptual strength has kept this system alive. It has been used in almost all types of applications with great success. Developers have proved that the concept is possible to digest and grown up accustomed to this standard specifically for procedural programming. As a result of this history, relational database systems are where most enterprise data ends up to be. If you are building an application that does not need to integrate with this data, feel free to use a different database model.

Why doesn't the ORM just persist objects to XML or a binary file(s)?

This may work for small databases when concurrent access to data is not required. A relational database allows concurrent access to data, locking, indexing (fast search), as well as many other features that are not in XML file. Take your bank's customer accounts database. If you go to withdraw $20 dollars, the system will need to search for your account quickly and find it among 4 million accounts. If you try to do this sequentially with a file, you will need to read about ((4 million+1)/2) records before you find a match - This is a lot of reads. Also, to update your account after you have done the withdrawal, you will have to write back the entire 4 million rows. Imagine this happening for every customer, then imagine this happening on 300 teller machines at the same time.

The following may be useful reads:

1-Relational Database Components

2-Benefits of Relational Data Model (see the bottom of the page)

3-Is The Relational Database Doomed

| improve this answer | |

For simplistic systems, with very basic data storage needs, I've often gone with a custom data layer which persisted to XML. However, to answer your question properly, we'll have to look at it from the perspective of most large companies.

Why are we even keeping SQL and a SQL database as the back-end?

One of the best answers that I can give you is that databases tend to outlive the systems that use them by far. Business data has meaning long after the front-end's technology has become obsolete. Refactoring or rewriting code (a task not taken lightly when it comes to large systems) is far easier than refactoring or redesigning a database. It's more than just not having the necessary tools and techniques (TDD / automated refactoring, etc), when you are working with existing data, there are many other factors to consider, such as downtime (changing things around in a 20TB DB takes a while).

Also - SQL, is a very good 4GL for accessing data. It abstracts away a lot of the complexity, and still allows for excellent performance (when things are done correctly).

Why doesn't the ORM just persist objects to XML or a binary file(s)? Why bother with all the overheads and difficulties of administering a database server?

If you think about what you are saying for a minute, you will realise that you are describing a database server all over again. The actual storage mechanism databases use is a highly optimised binary file. If you didn't use a DB server, you'd end up reinventing indexes, and then reinventing atomic transactions. Also, if you think administering a DB is a pain, you probably haven't tried to administrate millions of disparate XML files.

Is this the way of the future?

The DB world is certainly going to change a lot in the next few years, with "NoSQL" showing many benefits. The need for a mature, reliable way to store, retrieve and manipulate data isn't going anywhere though.

| improve this answer | |
  • I realise that a non-trivial storage system will require some sort of server-process to manage it, but what I am questioning is why the peristence mechanism should involve SQL and a database server which has its own set of data-types etc. This is an unnecessary layer - like translating English to French to German, instead of just going straight from English to German. – CJ7 Dec 14 '11 at 6:05
  • @CraigJ, In what way then is what you are describing different from the NoSQL databases others have already mentioned? – GrandmasterB Dec 14 '11 at 6:12
  • 1
    @CraigJ - I guess that depends on the perspective that you are looking at it from :) ANSI SQL (and it's data types) are a well established standard, certainly more standardised than the variety of languages we use to interact with DBs (C#, Java, Python, etc). If anything, ANSI SQL is the native storage format, and the code is the non-standard thing that needs translating. – Daniel B Dec 14 '11 at 6:31
  • @CraigJ If the persistence mechanism didn't have SQL and database server one would have had to emulate all that (storage system) functionality in each and every programming language that needs access to non-trivial storage system. – Srisa Dec 14 '11 at 9:18

Why are 'we' keeping them? I cant speak for you, but I need to do more with data than simply load and save isolated objects. XML files arent going to do me much good if there's no way to search them and perform calculations against the data they contain other than to sequencially open each one. How do you calculate a breakdown of, say, this years sales by month and product, if all your data is spread across millions of little xml files?

And, quite frankly, if you think managing a database server is difficult, by comparison, trying to track millions of files in the file system is a lot more complex!

| improve this answer | |
  • The ORM is meant to be doing all that for. – CJ7 Dec 14 '11 at 5:16
  • @CraigJ - So how does the ORM do it? Just because you don't see that functionality on your end doesn't mean it shouldn't be there for the ORM to do what it needs to do. – derekerdmann Dec 14 '11 at 5:22
  • @derekerdmann: the ORM should persist the object data to disk in it's own way. There shouldn't be the now redundant layer of SQL generation and translation of query results back to object data. The ORM should persist straight to disk, and not via the database Server. – CJ7 Dec 14 '11 at 5:45
  • I'd be suprised if there werent ORMs that do that. Storing objects that way is all good and wonderful, but only up until the point where you need to query the data in a way that spans objects. Then you're out of luck. Dont make the assumption that because your project doesnt need that, that others dont. – GrandmasterB Dec 14 '11 at 6:06
  • @Craigj -- ORM = Object Relational Mapping, of you drop the relational database its just a persistence layer. Very few people use a plain persistence layer because very few people find them useful. – James Anderson Dec 14 '11 at 9:45

You've touched on something that the NoSQL movement is based upon, that most systems (particularly simple web applications) are usually interested in just serializing out their content model objects to a key/value store, and that a lot of data scenarios can sacrifice duplication of data for performance/simplification gains with simple indexing. This does not mean SQL databases (and the ORM to talk to them) are going away at all however. There is still a great need for relational data and reporting on that data, of which NoSQL is not an appropriate solution for.

I'd encourage you to check out the following:

| improve this answer | |

The big problem is that OO Objects present a strictly hierarchical view of the data.

Take a typical OO "Order" object it consists of:

  • Some customer details.
  • Some Delivery details.
  • A collection of order items.
  • A product code.
  • A quantity.
  • A price.
  • Some tax data.
  • Total amount payable
  • Some payment details.

This is all well and good and can be mapped into six or seven relational tables via the ORM or one serialized object via an Object store, XML or plain text file.

This works fine for vanilla order processing. But consider what happens when purchasing want to know how many of each product were ordered today so they can manage their inventory.

  • With the SQL backend they can fire off a simple SQL query against the items table and get a total for each product in product id sequence.
  • With serialized objects they need a custom program to read all the orders, step through each order item, sort the items into the right sequence and summarize the quantities.

In any normal sales situation marketing will want to know sales volumes by product, area and channel. Warehousing will want products and quantities, accounts will want to know prices and taxes, credit control will want to track outstanding amounts by customers etc. etc.

Relational data bases were designed from the outset allow different views on the same data. By basing your tables on a conceptional Entity Relationship model that satisfies the needs of all the potential users of the data an efficient and flexible data store is created.

With the current crop of OO databases you can only design for a small set of use cases, other potential uses are excluded.

| improve this answer | |

A little while ago there was a similar question and someone posted this link, which I found quite interesting. Personally, I've used straight SQL as well as few ORMs and in my first job I've also spent quite some time optimizing SQL queries because what appears simple seems to always break when you start working with 1M+ rows. Although, I like working with ORM s and believe they have their place, I find myself sharing a lot of views with the author of that post.

| improve this answer | |

An ORM is an Object Relational Mapper, and as almost every relational database uses SQL there is not going to be much of a change as its really your only option.

If we were to take the point about persistence abstraction and go down that avenue rather than the high level ORM concept then yes it would be good to have a persistence abstraction which just lets you save your objects/models to a given data store, be it MySQL, MongoDB, FlatFile etc. The problem for this is that relational databases support relational data, so you can setup relationships between your objects, however most NoSQL based solutions are more concerned with persisting your object as a whole rather than normalizing it into smaller relational tables.

Anyway the main point im getting at is that ORMs were designed with SQL databases in mind, most NoSQL style databases do not use relationships (maybe graph dbs in a way) so an ORM without a relational database would be redundant.

| improve this answer | |

if I am programming in a language, then I want to use the language's native implementation of objects and data structures to drive my program

yes, exactly this. Only trouble is when dealing with data stored in a database, the language's native implementation is SQL. So technically if you're not using SQL when working with databases, you're doing it wrong.

I imagine the problem here is that you just don't get tool assistance like intellisense when embedding SQL into your code. My advice then is to use stored procedures, so your SQL is wrapped up in a nice API that the DB exposes to you (a bit like implementing a web service). Then you can use your ORM as a client access technology to call those sprocs with the added benefit of better decoupling, better security, and often better performance.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.