5

The system I am working on employs quite a sophisticated caching/preloading mechanism for external third-parties, and since it is built on a microservice architecture, I would like to extract the whole caching/preloading functionality from a feature microservice (say, booking/searching) to a dedicated one:

+-----+      +--------------------+            +----------+
| API | <--> | booking | internal | <--------> | external |
|     |      | service | cache    |            | service  |
+-----+      +--------------------+            +----------+

versus

+-----+      +---------+      +---------+      +----------+
| API | <--> | booking | <--> | caching | <--> | external |
|     |      | service |      | service |      | service  |
+-----+      +---------+      +---------+      +----------+

I am concerned with the potential bottlenecks this might introduce: instead of the booking microservice using an internal cache system that talks to a backend, it will communicate with a separate microservice for all get/set calls (that in turn talks to the backend). The bottleneck I see is mainly the added time for communicating through the extra microservice. Should this be a big concern? How shall I tackle communication between the booking microservice and the caching one (REST, message queues)? It should be synchronous communication, as far as I see things. What any other (hidden) bottlenecks can arise from this setup?

  • "I would like to extract the whole caching/preloading functionality from a feature microservice (say, booking/searching) to a dedicated one." Why? What if some services don't need to be cached and different services have different caching requirements? It's entirely reasonable to cache via reverse-proxy or on a service-by-service basis. – Eric Stein Apr 14 '17 at 13:02
  • 3
    The caching service caches external data from third-parties, not internal data inside the microservice architecture. That is, my booking microservice will not communicate directly to one of the many third-party ticket APIs; it will communicate with the caching microservice that in turn decides whether or not to forward requests to the external APIs. – linkyndy Apr 14 '17 at 13:07
  • Ah, sorry, missed the external bit. :) – Eric Stein Apr 14 '17 at 13:09
  • Include some info about the characteristics of the data your caching, and I will be better able to address your question directly – TheCatWhisperer Apr 20 '17 at 14:26
8

My suggestion is that instead of worrying about whether to use an external or internal cache, your first concern should be that your booking-service does not care whether or not your are using an external service.

That is to say, your booking-service should be caching against an interface with the concrete implementation injected in; it would not know or care if it was using:

  1. An external caching service (out of process)
  2. An internal cache (in process)
  3. A pass through cache which just went straight to the external service

At some point, you could even develop a system which used a smart combination of external and internal.

Once this behavior is properly isolated, you are then much more free to explore the advantages and disadvantages of each solution for your particular use case.

  • 2
    +1 ... I started an answer along these lines. But, this summarizes it pretty well. It's not all that difficult to abstract the decision away, setting yourself up to run benchmarks with various setups if issues occur ... And this answer is particularly good because it doesn't make assumptions about the OP's application, traffic patterns, and so forth. It safely leaves those details where they belong: In the OP's conference rooms. – svidgen Apr 14 '17 at 16:08
  • 1
    While I completely agree with you, it doesn't really answer my question. I've been through this phase and of course I will implement a flexible booking service. It's the next phase that I am concerned about, the "freely exploring" phase; some insights would help here instead of blindly trying out alternatives. – linkyndy Apr 18 '17 at 7:42
  • Here's the problem, without usage statics, it is difficult to say. In process caching is actually faster, but does not scale well. This equation is changed by the size of your cached objects, and the frequency at which cache misses occur – TheCatWhisperer Apr 18 '17 at 14:16
  • Include some info about the characteristics of the data your caching, and I will be better able to address your question directly – TheCatWhisperer Apr 18 '17 at 14:25
  • The data that is cached is the availability of different products sold through the booking service. These products come from various sources and the cached availability will also be used throughout the various services in the system. – linkyndy Apr 21 '17 at 13:51
2

You're indeed creating a serious bottleneck, and a single point of failure in your chain.

It'd probably be more appropriate to either set up multiple instances of your caching service as proxies for each of your actual services (maybe having each proxy proxy for a number but not all services to save some network ports). Another option is to have each service implements its own caching using a generic caching system (having each extend from an abstract service that creates and handles the cache for example, with the implementation classes not calling the external services directly but leaving that to a method in the abstract base which also handles the caching).

There's something to be said for both variants. The first of course means you might have less initial coding effort, but you have higher network load internally. The second has lower network load but you're going to have to do more work to rewrite your existing services to implement it (and for future services your people need to comply when coding them rather than when configuring them, do you trust your programmers or your implementation specialists more? Or are they the same people...).

  • 1
    As mentioned in the comment above, the caching microservice will act as a gateway to external third-party APIs, not to the system's microservices. So, the difference will be that instead of the booking microservice making a request to external API X (instead of a miss), it will call the caching microservice, which in turn will send a request to external API X (instead of a miss). – linkyndy Apr 14 '17 at 13:27
  • @AndreiHorak that's irrelevant for the amount of internal network traffic being generated by having the cache as a separate microservice. Your implementation services will still post every request to it, whether it goes external or internal (in fact I'd have it process internal requests as well, for consistency as well as making it easy to in time replace them with an external (or externally hosted) service by just changing one configuration. – jwenting Apr 14 '17 at 13:46
  • So I shouldn't worry about the overhead of communicating inter-process instead of intra-process for the caching functionality? What communication type shall I employ? Would async (pub-sub) be worth implementing instead of the traditional request-response (as in speed and fault tolerance)? – linkyndy Apr 14 '17 at 13:50
  • @AndreiHorak The inter-service communication in your system is all REST over HTTP (probably, that's what most tend to do). Alternatively, you could use a shared database, have one service write to it, start polling for a reply, and the other service poll for requests, then write the response to it. Similar network traffic, just different protocols. I'd just use HTTP, it works well enough. If you worry about comms overhead on your internal network (or between your hosting centers...) you'd best use my second scenario of having each service manage its own cache. – jwenting Apr 14 '17 at 14:00
  • That way you'd have no traffic between your services and your caching service as there'd be no caching service. Note that the first option in your edited question confirms to my second option :) – jwenting Apr 14 '17 at 14:01
2

I only see three good reasons why the caching should be extracted:

  • Multiple internal services interact with the same external service. If all your services would share a cache, this could be a net benefit as there would be more cache hits, and fewer duplicates between caches.

  • Interacting with the external service is fairly complicated, and you want to offer a simplified API to your internal services. Here, the caching service would rather be an “adapter”. However, it is not really necessary to turn the adapter into a separate service, as a native library would probably also work.

  • Due to resource requirements, the cache needs to run on different hardware than the booking service. You would like to scale the cache independently from the other services. E.g. if both your booking service and the cache are RAM-constrained, two medium-sized servers might be cheaper than a huge server that has enough resources for both.

  • 1
    The first argument is true, which kind of implies the second one. The last argument will apply, but not immediately. Still, my question was not about whether to split the services, but how to efficiently communicate between them. – linkyndy Apr 18 '17 at 7:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.