Design question - storing data twice

Question

I have a polyglot database solution that includes the following components:

• a management interface that consists of a web application and a document based database (either rethinkdb or mongo)
• a routing service that is a key/value store database... with a simple REST API that allows CRUD functionality.

The routing database is populated by the management application. My question is this. Should I store the routing information in the management application, as well as the routing service database? or just once?

Arguments for storing data twice:

• possibility of recreating the routing database if something were to happen and we lost it.
• when it comes to creating routing reports, I can ease the burden on the routing service by reporting off of the management application. (the routing service will be used a lot more than the management application)

Cons for storing twice (aside from the obvious one about using more resources)

• can potentially end up with discrepancies between the management database and the routing database. Will need to have a script of some sort to check for these discrepancies from time to time.

I'm not sure what else to add to my pros / cons list. If you have any comments I'm all ears.

Answer 1

I would say store once.

As for the benefits you've listed about storing twice, I'm not sure those are really benefits. If anything they are solutions for problems that already have better solutions.

• Being able to recreate a database - that's what backups are for. If your schemas ever diverge because the API needs something more than the management app, your chances of a successful recreation drop considerably (and probably approach zero).
• Easing stress on the API - good idea, but that is probably better solved by taking advantage of load balancers, caching and the scalability of most document databases. If these things can all scale pretty well, scale them when you need to. Don't solve a problem you aren't sure is actually a problem yet.

The disadvantages of storing twice, IMO, feel like pretty big ones.

• Dealing with data discrepancies can be a huge pain. It means a lot of developer time spent figuring out why something a user just saved isn't working.
• Continuing with discrepancies, when something is broken, which system is right? What are the costs of the management app (and its reports) being wrong? What about if the routing API is wrong?
• More external dependencies for your app. Now you have to be prepared to deal with having multiple databases be unavailable. How do you handle when one is down and another is up? Save it one place and fix the other one later? Or just crash and burn and hope the database comes back soon?
• Kind of piggy backing on the previous point, now you've added another bit of complexity to the app. Now you have two databases to deal with, an external script or job that has to be maintained and run, presumably maintaining another server / db installation, etc.

Answer 2

Store twice.

The management interface should keep a history of changes, audit info and all the extra meta data you don't want to pollute your routing db with