9

I'm designing a system from which I will synchronise business data from the mobile device (that have an embedded application) that generates data and sends it back to the server. Each line synchronised generates a specific business log in the database.

If what I synchronise generates data with a date (within the sync data) inferior to the last modification date of my business data, I must ignore it and just add the log in database. Once the uploaded data is processed, data is fetched from the database and downloaded to the device.

Because of this download right after writing, the synchronisation must be synchronous. It's possible still to have a reader/writer pattern if something like this is worth enough to replace my existing solution. The more important thing is to be able to download up-to-date data. That data is fetched as a whole, there is no diff implemented at the moment (it may come later but that won't be a problem).

I may have multiple synchronisations on the same business object running, it's unlikely but can happen and I prefer to be able to handle it. The synchronisation is expected to last for some seconds but not some minutes, unless using the embedded mobile application without resync for some days.

The volume of data synchronised is not expected to be big, neither the synchronisation process.

So I end up using a mutual exclusion on my method of synchronisation, more precisely, I'm using Java and I put a synchronized on the writing method not the whole synchronisation process to not block read-only synchronisation.

I would like to know :

  1. If this way makes sense? As long the volume and time of synchronization process are still acceptable.
  2. In a general way, what concepts should I look at. Bonus: if there is any implementation of these concepts in a Spring module.
  • What causes the offline? I mean when the device is offline it means that has no access just to the server or it doesn't to Internet either? – Laiv Jun 3 '17 at 20:20
  • It hasn't access to the internet. Or not often. – Walfrat Jun 7 '17 at 7:11
  • If you have multiple clients/servers synchronizing, you must first decide on data mastership in case something differs. If you consider intermittent connections only and multiple clients, there definitely is no way to do this incrementally. – tofro Jan 3 '18 at 14:31
  • @tofro data mastership is easy to determine in my case so it's not a problem. However why wouldn't be possible to do it incrementally with intermittent connections ? Shouldn't I be able to just use a last sync date ? The only problem in my case using such a date would be how to know that a data that is currently on my device has been moved and should be deleted on the device. – Walfrat Jan 4 '18 at 8:53
  • From your description, I have understood that the same data item can be changed on the server, or on one or more multiple clients. How would you three-way-sync a data item that went to mobile #1, then was changed on the server, then went to mobile #2 and was changed there, then mobile #1 connects (with a disconnected mobile #2)? – tofro Jan 4 '18 at 9:53
1

One approach that I've been investigating for awhile now (with some success) to have client data sync up with server data, without relying on dates (which might be unreliable) or synchronous requests, is a combination of JSON Patches (perhaps POJOs in your case) and event sourcing.

The basic idea is that instead of storing the current state on the client and the server, the client and server store a list of changes, and message each other through either events or patch requests.

So instead of having the client send all the data plus a date to the server, the client sends an event, along with a revision number that corresponds to the last time the client thinks the data has updated. Something like this:

Server.send("MODIFY FOO", 3);

Once the server gets this event (asynchronously), it reconciles it with other events it may have already received. For example, it's possible that another client working with the same data might have already modified some things, and now the revision number on the server is at 5. Thus, this revision will need to be applied before the last 2 were applied, and all clients will need to be notified of this change.

Once the server finishes, it notifies all the interested clients of the changes have been made, and the new current revision number. The client then applies these changes and updates its internal revision number.

Your mileage may vary, but I hope that helps.

Edit: Another name for this approach, or a variation of it, is called message queuing, as mentioned in this related question.

  • The system can be offline (no access to the server) for some days and must register the date when the event occured, not when the event was synchronized with the server. That's why i have to use dates. I have a revision number for optimisticLocking too, but for the same reason, 2 device could download the version X of a POJO and when they sync later each send events that must generate a version X+1 and X+2. And the devices can't communicate with each other. – Walfrat Mar 10 '16 at 7:55
  • The revision number in this answer is always generated in server, client send old revision numbers, it does not need to know about other clients as revision increment is not its responsibility. This answer fails to mention conflict resolution, which is the most important part of proposed solution. – Basilevs Feb 3 '17 at 19:30
0

First issue is using dates as a way to synchronize data. I'm really sure I didn't get all the details of your solution but I would say that:

  1. Are the dates generated on the mobiles ? In this case, are you sure the app running on the mobiles will always use the correct dates ? What about a malicious user that somewhat is able to change the system date on his / her mobile device ? What about users in different timezones ? As @jeffrey told you, maybe it's not the best approach to rely on dates generated on the devices.

  2. If I understood properly, you use Optimistic Concurrency Control. I don't see anything that is intrisically wrong in your approach.

  3. This question is about implementing Optimistic Locking in Spring. Maybe you can find some inspiration in it.

  • I need to record the date when the action was executed, as such i'am stuck with the mobile's date. The mobile date is frequently resync with the system. Only registered deviced can sync with the server. As for security it is another concern left for now (X509 device authentication,...) – Walfrat May 22 '18 at 13:19
  • Date could be recorded as yet another data field, no need to treat it as synchronization token. – Basilevs Jul 22 '18 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.