Data autonomy using files in a micro service application

Question

We are rebuilding an on premise application using microservices and are wondering how to implement data autonomy given certain constraints that we are facing:

The application is built around a set of files that it needs to keep in sync with the cloud service. We have very limited bandwidth available, and so we have to make use of the existing functionality were we send a CRC of the current files and receive a patch to update the contents of the folder.

• Is it reasonable to give the services direct access to a sub folder that contains the data it should own?
• Is it reasonable to have a separate service in charge of patching all the data, or should we strive to make each service responsible for patching its own data?

Here is a contrived example of what we are building:

Answer 1

The fact that you keep your data in files, and that they are updated from an outside source adds a wrinkle to this problem.

If you have say 2 instances of the weather service, having them both read and update the same file if going to run into locking issues.

Additionally you will want to only have the updating check run once, not once per instance.

Therefor, I would advise hiding the files behind another service, which can deal with caching, updating, avoiding locks etc.

If the files in the set can be divided up into their retrospective services however, you could do that and have multiple of these file protection services, one per group of files. This would be more microservicey

Or you could separate the update service and use a database to store the information rather than the file system, which would solve the locking issues.

A push messaging or queue system could solve the update issues without a separate service.