How to minimize networking cost of distributed game database?

Question

We've been working on a networked online game project which will run in near future and looking for a satisfying solution for our distributed game database. We didn't take networking costs into account so far.

We have following constraints:

1) We planned to use MySql but we can review alternatives if offered solution makes sense.

2) We planned to host multiple game servers on different geographical regions such as USA, Europe and Asia.

3) Every gamer usually connect to the nearest game server and distributed database node in order to reduce networking costs and lags.

We've been thinking to develop a logic in one of our server side layers that will check the user location he wants to connect from. Then, replicate user specific data from one database node to another if the check operation results that the gamer attempt to connect from a different region than he did before.

We wonder that is the correct way to do that?

Of course we have to think disaster cases. For that purpose, what can we do if one of the distributed database node fails?

We don't think to replicate all data between database nodes because it causes high networking costs.

Thanks

Answer 1

I would avoid any ad-hoc, on-demand, database replication. Ideally you want a single consistent, logical user database across all of your nodes, for disaster purposes.

@Wardy's recommendation of a service bus is a good one.

I won't discuss different cluster architectures, but will address the issue of network bandwidth and transaction volume.

With high transaction volume systems, you do often run into an issue where there is so much DML (or transaction volumne) that you can't do full transaction based replication; one way to reduce overall replication bandwidth is not to replicate transaction logs, but to do snapshot based replication in a delayed, larger granularity frequency than the average record update frequency. I call this coalescing or collapsing the stream but I don't know if those terms accurately describes reducing the volume in this way.

For example, if your average active player record updates every 15 seconds to its local database node, you can setup a 10 minute replication cycle, and coalesce 40 average updates per record into a single replication pull. Only records that are modified since last pull will be transmitted, and you'll replicate a single record even though there may have been 40 updates since the last synchronization. This reduces your potential network volume by an obvious ratio.

In this model, things like row-level trigger based auditing doesn't work because you don't get all transactions, just the latest, so you must work that out in your design.

As to implementations, a good example is Sybase iAnywhere Mobilink. But I've also seen it implemented in-house on top of various database flavors.

Answer 2

Sounds like a typical use for a service bus based solution.

I solved exactly this problem but for keeping running servers live and interconnected and in sync with each other in my MMO server framework.

The simple answer is "don't try and sync the data once its in the database use an event and handle it in multiple locations".

It's basically like taking the same approach as cloud computing does by having several clouds you either need a fat pipe on a scheduled basis or a means of seeing an incremental change as it happens.