1

I'm developing a WPF application whose core functionality involves creating a large object graph (often tens of thousands of entities) which the user can modify certain parts of, then save to a database. A graph can also be subsequently retrieved, modified and saved again. It's feasible that a user can create dozens of these graphs per day.

The app also provides the user with different ways to search the entities in the database (predominantly text searches on various fields), presenting the search results to the user and allowing the user to select one, which will result in the relevant entity graph being retrieved in full and presented for the user to view and modify as above.

Currently I'm using Entity Framework and SQL Express, but I'm uneasy about certain aspects of the architecture and design, and the client isn't keen on having to install SS. I've only recently come across the concept of NoSql databases, and sounds like they might be a good fit for what I'm doing, but I have a couple of questions.

First, I'm assuming performance wouldn't be any worse than EF when reading or writing one of these object graphs?

What about my app's search functionality? Will a NoSql db support this sort of thing, and what would the performance be like, bearing in mind the size and quantity of "documents" that I'm likely to have.

What about lookup (reference) data? Would I duplicate such data in each NoSQL document, or would I keep it all in a single NoSql document and store their IDs in the main documents?

Finally, any recommendations for a product? MongoDb and RavenDb seem to be the main OSS contenders for Windows.

2
  • 4
    There is no such thing as NoSQL. Only lots of different new database technologies which haven't got much in common.
    – Philipp
    Commented Sep 4, 2013 at 19:11
  • RavenDB is not free-as-in-beer, and you may want to look at the licensing terms before you pursue that much further. You can either use it as GPL (which means your software also has to be GPL) or you can pay not-so-cheap licensing fees to ship/use it as part of a proprietary product.
    – Aaronaught
    Commented Sep 5, 2013 at 1:07

4 Answers 4

Reset to default
6

This is simply the wrong question to be asking.

"NoSQL" doesn't refer to a specific database, it refers to an entire superclass of databases including document databases, distributed key-value stores, graph databases, and object databases.

Speed is generally the least important factor in making decisions about data storage, no matter what some people might tell you. A SQL Server table with a billion rows can do key and index lookups at about the same rate as a MongoDB collection with a billion documents or a db4o database with a billion objects. The exception of course is if you can do sharding, in which case you'll want a product that supports it, but if your users are antsy about installing SQL Express then rest assured they'll run for the hills if you tell them to grab 200 old desktop PCs and shard an HBase instance across all of them.

Do you need full-text search? The industry standard for that is Lucene. It's not really a database by itself, it's something that's bolted onto other databases and sometimes packaged more nicely by tools like Elastic Search or Solr. Most SQL databases also have some form of full-text search; it's generally slower and inferior to Lucene. Some NoSQL databases have full-text search (RavenDB, for example, actually uses Lucene) but most have either no support or are in a very primitive stage.

Do you need to store very deep hierarchies of objects all at once, hierarchies that always have a single well-known aggregate root? If so, then document databases like MongoDB or CouchDB would work well for you. But if you ever need to change your hierarchy, or find that you need transactional consistency (and not the "eventual" kind) then you're in for a world of hurt.

Does your data consistent of multiple related entities that each have independent lifetimes? If so, then a relational database like SQL Server or mysql is by far the best choice. Relational databases allow you to defer or ignore many difficult modeling decisions and in general you'll only have to model once, as compared with document databases or key-value stores where you may have to frequently change your model or maintain multiple parallel models in order to solve various different use cases. If you want to keep things simple, you definitely want to stick with SQL.

If you need an embedded database, consider SQLite. It's not as powerful as SQL Server but it's fast and easy to use and easy to deploy, and you'll find the syntax to be mostly familiar.

Incidentally, if you're really that worried about speed (and by the sounds of it, your needs are way too small to justify any such worry) then you might want to look at the benchmark done some time ago by the ServiceStack guys. Entity Framework comes in dead last, and by a pretty wide margin too. NHibernate is probably the best choice when you balance out the mutually exclusive requirements of compatibility and performance. Although personally I prefer not to use any ORM at all, and you definitely won't be able to use a pretty little ORM if you switch to NoSQL. Good luck learning the eleventy billion Redis commands if you've got no prior experience with the product.

4
  • I started with SQL and EF as this is what I know best, but I'm finding EF to be painful with regards to loading and saving one big graph. In answer to your questions, yes it's a very deep hierarchy with a single root that will be retrieved and saved all at once. My only speed concern is with regards to the search functionality. Not fully understanding how NoSql works, I'm assuming they employ some kind of indexing rather than have to load each graph in order to search. The users wouldn't want to be waiting "seconds" for search results to be returned.
    – Andrew Stephens
    Commented Sep 4, 2013 at 21:16
  • @AndrewStephens: It doesn't seem as though you've fully absorbed what all of the answers here (including mine) are telling you: There is no "how NoSQL works" - it's a self-contradictory phrase. Some databases, like MongoDB, use indexes. Others have no concept whatsoever of an index or even an entity - they're like giant hash tables and you only get primitive types as values. Most graph databases have something vaguely analogous to an index but they don't work the same way (and, contrary to what some answers are suggesting, "object graph" has nothing to do with "graph database").
    – Aaronaught
    Commented Sep 5, 2013 at 1:00
  • @AndrewStephens: Moreover, I've never had a SQL query take anywhere approaching a second to retrieve less than a million or so rows with proper indexing - any slowdown is more likely to be due to the network or disk speeds associated with actually retrieving the data, something which no database engine can fix (unless you throw tons of RAM at it, in which case any of them can fix it). Let me say this again, NoSQL does not equal "fast", just "not SQL". And if you're worried about search times then use a full-text search engine, which is completely irrespective of your database.
    – Aaronaught
    Commented Sep 5, 2013 at 1:04
  • +1 Although one example does by no means settle it but only last year did I write my customised full text search engine by exploiting the text structure than go for a Mongo solution. It outperformed on memory benchmarking and matched on time to query but the biggest wins were in the flexibility of the queries that I could run without changing schema. Point of this story is --examining your data can bring about better wins sometimes.
    – Apoorv
    Commented Sep 5, 2013 at 12:18
5

There is no such thing as noSQL. There is only a whole bunch of new database technologies with completely different philosophies and use-cases, and all they have in common are things they also have in common with SQL databases. That means when you plan a project and are unsure about the database technology, you need to evaluate each noSQL database individually.

When your data is based on large graphs, it sounds like a perfect use-case for a graph-oriented database like Neo4j.

Document-oriented databases like MongoDB are generally not a good fit for graphs, because they don't support connections between documents very well.

2
  • +1 Maybe the question should be "is there any noSQL database suitable for me ? and which one ?"
    – Tulains Córdova
    Commented Sep 4, 2013 at 20:11
  • Some of these technologies aren't exactly new, per se. Column-oriented databases have been around since at least the early 80s, with Teradata. Object databases go back to the 90s with Smalltalk. Lotus Notes would technically qualify as a document-oriented database. Graph databases as a concept have been around I think almost as long as SQL. These things aren't new, they've just come into vogue during the past 5 years or so after people heard that Google and Amazon were using them.
    – Aaronaught
    Commented Sep 5, 2013 at 1:16
2

Design First, Shop Later

You should design your system before you start shopping for specific implementations. Specifically:

  1. Design your data structures based on how you plan to search and retrieve data.
  2. Evaluate the efficiency and performance of comparable solutions that implement your design requirements.

There's really no substitute for benchmarking various solutions on a representative corpus of data, and no one else but your organization can evaluate the various trade-offs for your particular project.

Graphing? Then Graph!

Choosing the right data structure to represent your data is essential to preventing premature aging in programmers. Selecting the right models for searching, storing, and retrieving your data is very context-dependent, but it's the essential first step in designing a new system.

I've never found a practical need for it myself, but if your fundamental data structure is a graph, why not take a look at a graphing database solution? Neo4j seems like it's designed to address your problem domain, but I can't shill for it since I've never used it.

Even if Neo4j is not the right solution for your project, you should certainly investigate all the options that make it easy to work with your core data structures. A key goal is to reduce the number of transformations you need when converting data between input and output formats, so keep the focus on how you will use the solution.

1
  • An object graph has very little to do with a graph data structure. The data stores most closely aligned with object graphs are object databases, with document databases coming in a distant second (JSON can represent just about anything). Of course that's assuming that the real need is actually to serialize and store an entire object graph, which IMO hasn't really been properly established.
    – Aaronaught
    Commented Sep 5, 2013 at 1:20
0

Its quite likely that a NoSQL database will meet your needs better than an SQL database, but all that does is increase your choice options.

Pretty much every type of database can store your data set, but what really matters is what kind of searches you need to do. The different database types have very different abilities in this area, and that is what will help you identify the suitable types of database to use.

This link contains a good list of different NoSQL products grouped into their loose behaviours.

What I would say though, is that most of these products are not targeted to the windows environment. This means that you would need a linux database server and that you have a seperate application server hosting the windows application.

Not the answer you're looking for? Browse other questions tagged or ask your own question.