Problem overview:
I'm creating a spring application just for learning purposes. I would like to create a microservice just for files, which:

  • at the beginning would have only two basic endpoints upload/download a file (used by other microservices, not by a user directly)
  • create db only for that microservice with thefile table. At least at the beginning.
  • store files in a cloud (f.e. Amazon S3)
  • store meta-information about the file in the file table. By meta I mean:
    • Url to file in the cloud
    • Name
    • Size
    • Type (a directory or a file)
    • Extension

Example flow:

  • a microservice A sends an image: user_attachment.jpg to the files microservice upload endpoint.
  • server does validation, secuirty check etc.
  • sends the file to a cloud storage f.e. Amazon S3
  • saves meta-info about that file in a local table called file. F.e. it adds: INSERT INTO file (url, name, size, type, extension) VALUES ("http://aws.amazon.com/testapp/pictures/2018-09/picutre.jpg", "picture", 34233, "FILE", "JPG")
  • send meta-info back to the microservice A that initialized upload operation.

Uploading to cloud would be in a different thread, asynchronously and maybe even with a messagebroker help.


  1. Is it a clean/good-design-patterns-compliant approach ?
  2. What is good/bad about that approach?
  3. How could I do that in a better way ? (don't want to store files (content) on the machine where application is running [locally] )

This example is simple. I believe in commercial/big projects design might have similar assumptions/foundations. Thus let's assume that there is multiple tables in the file microservice, and both their structure and microservice logic are advanced.

How does file-management-microservice in commercial application might look like according to good design-pattern rules ?

  • 3
    Why would you do that? It doesn't seem like you're adding any value compared to using S3 directly, while the cost of doing such... proxy is rather high. So what's the point? Sep 9, 2018 at 19:38
  • I wasn't precise enough, sorry for that. file microservice wouldn't be used by users directly, but indirectly via another microservices (let's call them A, B and C). Yes, I could use S3 directly in all of them. In my question I just wanted to know how it's done in commercial/big projects. I believe the foundation of files-managment-microservice is designated in a similar way (but logic and db tables are obviously more advanced). By my question I just wanted someone to show me how file-management should look like according to good design-patterns. Sep 9, 2018 at 20:13
  • If it’s just for educational purposes, why not just store files locally?
    – RubberDuck
    Sep 11, 2018 at 9:33

2 Answers 2


A goal of a service in SOA is to perform some logic which, otherwise, would have to be duplicated among other services. That logic is, in turn, the reason for the service to exist.

In your case, you add no value through the service. Any other service could easily access S3 directly, given that S3 is itself a service (it is not your service, but that is irrelevant here).

Not only there is no added value to the intermediary service, but its cost is rather prohibitive. You may imagine that you can draft one in a few hours, which is true. However, in order to have something that works reliably and handles all the edge cases, you have to deal with caching, redundancy, interrupted uploads and recovery, large files, timeouts, brute force and DDOS, special characters in parameters, permissions and IAM, and dozens of other pretty things.

Therefore, you're about to spend two or three weeks of development and possibly one week of debugging (YMMV; I don't know your experience in HTTP, SOA and the technology you want to use to create your service, but even if you're highly experienced, you shouldn't expect spending less than two weeks on that, which is a lot).

I just wanted to know how it's done in commercial/big projects.

Companies pay on average $700/day for a highly experienced developer, so given my estimate of two weeks, you're at $7,000 for a task. Commercial projects are about money; when there is an opportunity to save an amount even as small as $7,000, they'll do it.

Additional to this direct cost, there is a cost in terms of hosting and code maintenance. This service will have to be maintained for years, adding to the bill much more than the original price. Again, all this money wasted without the clear understanding of what could such service save to the company. It doesn't save bandwidth, nor other resources. It doesn't reduce the amount one will pay to Amazon, so...

This is not all. The cost of maintenance of all the projects which depend on the intermediary service will also increase. If a service:

  • Has to be patched and the patch requires an interface change,
  • Has to be moved to another location, with a change in its URL,
  • Is down, requiring to have a circuit breaker to ensure the client service doesn't go down in turn,
  • Is deprecated, requiring to migrate the client to something else,

the immediate and unforeseen maintenance is required and is usually costly as well. Now, it is much more likely that those four things happen to your service than to Amazon's S3. Not because you are a bad developer, no. But because Amazon's scale is slightly different than the scale of your service, which means that they have much more workforce to pay to ensure the clients can rely on S3.

Finally, many developers have prior experience with Amazon AWS (and possibly S3). This means that when you hire a new guy, he can easily understand how a service is storing files if it uses S3 directly. If you add a level of indirection, this benefit is lost. And, more importantly, every time someone has a problem with the storage, he would need to ask himself if the problem comes from the client service, from your intermediary or from S3. This adds to the debugging time.


Is it a clean/good-design-patterns-compliant approach ?

No. Add services when they add value. Don't make things more complex than they need to be (KISS) and, specifically, don't add layers which bring no benefits.

What is good/bad about that approach?

What is good: the fact that you provide an interface which is much simpler compared to S3. For developers unfamiliar with AWS, S3 can be quite complex.

What is bad: already told it above.

How could I do that in a better way ? (don't want to store files (content) on the machine where application is running [locally] )

By calling directly S3.

  • 2
    Would the value of abstracting away the service provider be enough added value to justify the service? While cloud file services offer about the same features, it is an outside service that can creep through your entire infrastructure if you aren't careful. And if you don't centralize this within your ecosystem, changing cloud file providers can be a traumatic change, can't it? Sep 10, 2018 at 17:27
  • Given the requirements, such migration won't be very difficult to perform. Consider the other scenario. Services A and B use the intermediary service which abstracts S3. One day, team A decides that it needs a feature which is only available in Azure Storage. Team B, however, doesn't want to pay more for the storage. So now, the intermediary has to support S3 and Azure Storage at the same time, and somehow provide features of Azure Storage selectively for clients who have chosen it. Sep 10, 2018 at 18:03
  • 3
    I agree but the real advice i would give is to develop such microservice to be directly consumed from clients. The benefits of a "storage microservice" are clear: hides the underlying storage apis, it can add authn/authz features, it can track content metadata, implements "gargbage collection" mechanisms that remove files when they're not referenced from other microservices. Mar 2, 2020 at 23:35
  • Haha. The funny thing is this answer is more or less right, but that doesn't stop people from rebuilding this kind of service again and again. I had to build a service almost exactly like this recently. I pleaded to not have to do it, but the client was paying so... Oct 12, 2020 at 5:48

Amazing explanation by Arseni.

One advantage I see of this intermediary service is if it provides the caller (micro service or end user) the ability to choose the storage to be used for their use. I don't know how much of a practical suggestion that is, but I often save my important document across multiple cloud storage vendors for fear of losing it on one. So, this intermediate service could provide the ability to choose which storage the caller wants to use (optionally) or else default it to the cheapest that suits the caller's purpose.

  • In that case, it would be enough to return a descriptor that specifies where to upload the file and a client lib that can satisfy understand this. Jan 29, 2020 at 13:04
  • In this case the client part that is responsible for uploading the file to for instance Amazon should have a simplified abstraction that can be easily drop-in replaced with Azure, Google or any other cloud service, or even your own Microservice should you ever change your mind in the future.
    – Arwin
    May 3, 2020 at 17:25
  • You can also add claims or acls with better control than raw blobs stores low s3 or azure storage. And still get the abstraction from the underlying provider. Apr 4, 2023 at 10:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.