I was wondering - too many microservices usually cause testing and devops overheads + it makes the system more complex.

My service is something that handles a large load (from a data stream). It needs some database data from time to time.

There will be also a seperated process that needs to keep feeding the database with data from external source.

I initially thought of 3 services:

  • The service that manipulates and handles the data
  • Service that feeds the database
  • REST API service that will be used also as internal but also as external (eventually for UI), and internally for requests from the services.

I have several cases i'm considering:

  • I've read that a CRUD service is an anti-pattern - so i'm not sure about #3.
  • The first manipulation service needs a special scale algorithm, and the REST API that will be used externally also, requires totally different scale (much lower)
  • Service #1 and #2 actually share the database (or use the REST API) (though they are in the same domain) - it sounds like it's an anti pattern.

My main question is - Should i combine some of the services above? having mentioned that scale requirements are different.

  • Are you sure that the second bullet is really a service? What calls it?
    – JimmyJames
    Sep 10 '19 at 15:59
  • Actually #1 is questionable too. "Manipulates and handles the data" -> for what purpose?
    – JimmyJames
    Sep 10 '19 at 16:02
  • The first is part of an ingestion pipeline, there's a large amount of data that comes from various sources and needs to be handled, then we are persisting it. The second bullet is an independent process, that is syncing let's say configurations and data from our monolith
    – ArielB
    Sep 10 '19 at 17:44
  • Not everything needs to be structured as a service. The way you describing it, I wonder if it is more like a 'job'. Are these sources pushing data to you or are you pulling data from them?
    – JimmyJames
    Sep 10 '19 at 17:55
  • Data is pushed to a q and the services read from the queues.
    – ArielB
    Sep 10 '19 at 18:59

Microservices are not like namespaces. You don't use them to separate logical parts of your architecture as you would with a namespace. So a single dev, looking the business domain and sort of judging what "belongs together"-- that is not an optimum design approach (even though it seems to be rather common with engineers and managers trying to keep up with development trends).

Microservices are units of deployment. The point of a microservice is that it can be versioned, deployed, scaled, and managed independently of the other services. So some reasons to separate things out:

  • Functionality that has unusual load and may need to be scaled differently. For example, you data ingestion service might handle very large files very slowly, and therefore have different hardware requirements to perform at an acceptable level.

  • Functionality is likely to version at a different rate. For example, internal services might be subject to rapid change while services exposed to the public occur on a slower schedule. Separating them into separate deployment units would help keep high uptime while not tying you hands with the internal stuff.

  • Functionality that requires different security. A public service exposed on the internet may have different security mechanisms from a service exposed on your intranet. So they get deployed to different servers that are behind a different sort of firewall.

I suggest reconsidering your division of services based on the above.

The service that manipulates and handles the data

I don't know to respond to this since to my knowledge all computer programs manipulate and handle data.

Service that feeds the database

Sounds like this might need different scaleability from the rest-- separate it into its own service.

REST API service that will be used also as internal but also as external (eventually for UI), and internally for requests from the services.

Sounds like the internal and external services have different security requirements-- separate into their own services.


The last candidate seems to be an API gateway: It's not really providing its own services but only routing/converting service requests. It should therefore rather be viewed as part of the infrastructure supporting the microservices and not as an independent microservice in itself

In absence of additional insights, it is not clear how much the two remaining microservice candidates are coupled:

  • The shared database could suggest that it's about one single microservice that was artificially broken down. But this is not a sufficient argument, since shared database is a valid microservice pattern.
  • The fact that one microservice feeds the database and another occasionally reads it, raises the question of the necessity of the dependence to a shared database. Couldn't the feeder have its own database, and provide the relevant data to the processor ?
  • Finally, both candidates could as well appear to be loosely coupled microservices, for exampl,e each performing some jobs in a larger streaming archtiecture.

So up to you to review the arguments in favor of a partitionning the processing into different microservices.

  • API Gateway only routes to the Restful controller - the question is where is the logic of the rest API is written? in a seperate service? in one of the other core services? - about the "provide relevant data to the processor' - not exactly, eventually the database will contain data that is consumed also by the UI, and the processor needs to "ask" for data - should i create a RESTFUL API to the feeder? (in the future the feeder will be obsolete as it's job is to sync between the old system and the new system).
    – ArielB
    Sep 11 '19 at 9:59
  • @ArielB the microservices should be loosely coupled. If you split the rest api apart, the parts would be tightly coupled. This would be against the principles of that architecture. So each microservices shall have its own rest api, unless you let the api gateway do the protocol conversion (e.g. offer rest, but translate request to message queuing, for example for your feeder process).
    – Christophe
    Sep 11 '19 at 10:20
  • What about REST APIs that are intended only for UI purposes and customer APIs?
    – ArielB
    Sep 11 '19 at 12:39
  • @ArielB rest api for client ui purpose seem to be good candidates for a gateway (you could even go for specialised gateways per client technology). You may find some relevant arguments here: docs.microsoft.com/en-us/dotnet/architecture/microservices/…
    – Christophe
    Sep 11 '19 at 13:14
  • I understand, i think we don't understand each other - the API GW is fine, but there's the REST API controller (WEB API container in the sketch) - i'm wondering if the sole purpose of the web API container is retrieve info from the DB, is this a microservice? because if not, i will have to attach it to the microservices that perform processing - which sounds weird
    – ArielB
    Sep 12 '19 at 9:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.