I am trying to understand the architecture of the system described in this patent about aggregating and analyzing confidential data: https://patents.justia.com/patent/20180089196.

The general schema shows three different backends in this system: Confidential data backend, Backend queue and ETL backend: enter image description here

Why are there multiple backends in this system and what is their purpose ?

closed as unclear what you're asking by gnat, amon, Greg Burghardt, Bart van Ingen Schenau, nvoigt Dec 7 '18 at 15:07

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    A separation of concerns would be my immediate guess. – TommyBs Dec 7 '18 at 11:46
  • Big volumes and security ? Could you post the reference to the patent ? Maybe there's more explanations in the claims – Christophe Dec 7 '18 at 11:49
  • Answering this question would require us to read and understand the patent, any other answer will just be a wild guess. See also Discuss this ${blog} on our Meta site. The current answers even state “Perhaps the rest of the patent explains more” … “However, we can read the diagram and guess.” Note also that patents are notoriously unclear and overly general. I bet the patent says those backends could be part of one or more centralized or distributed systems. – amon Dec 7 '18 at 13:03
  • @amon: Actually, the patent even says that the drawings are merely examples and thus not really part of the claim at all. – Jörg W Mittag Dec 8 '18 at 9:20
  • Jorg,yes,because of that I am having trouble to understand what is going on. – MikiBelavista Dec 8 '18 at 11:07

In many senses, the question seems too broad to me and it's going to be hard to give you an accurate answer to the question What each of these backends really does?. However, we can read the diagram and guess.

A key concept here is separation of concerns. A secondary one could be distributed computing. For brevity, let's focus on the first1.

What you see is a complex backend comprised by several processes each of which is specialized on a specific concern (job, task, whatever).

Think in any business which nature involves treating the data through several stages each of which happens at different times, with different requirements, with different needs of performance, reliability, consistency, etc.

Instead of a monolithic "backend" doing all the above mentioned, the whole business is dissected so that each part is performed in different stages by dedicated, different and interconnected processes.

There could be many motivations to go with this architecture, from economical to technical. The one that caused our diagram we can not say, but we can guess (again).

Looking at the diagram the presence of a queue might suggest a need for high performance, the ETL suggest complexity on the data intake and the workflow suggest business rules to be performed in a very specific way, time and order. The API suggests that there could be clients monitoring and managing the data. The Confidential Data Backend seems a front-controller to allow clients to send new data. Altogether make me thing in a system where there could be potentially many clients sending data and monitoring its state concurrently.

Related links


1: Looking at the diagram we cannot say whether the system is deployed as distributed or not. But the separation of concerns is quite evident.


Analysis of the system components:

  • a main backend called Confidential data backend: it acts as a facade to the external world, receiving front-end requests and communicating with external services through a sub-component called Data bus. It also interacts with a database.
  • a Backend queue : the queue receives all its input from the Data bus but delivers output only to the ETL. This unidirectional communication suggests that its purpose is to implement an asynchronous processing. It may be used to absorb peaks (for example mass data received from the external services) or to allow the ETL to distribute the load between several processes/threads (for example for processing big volumes).
  • an ETL Backend: it processes data from the queue and performs Extract-Transform-Load to update/feed the database

First conclusions:

  • the multiple backends seems to implement a pipeline approach, with the purpose of handling big data volumes (loading peaks, mass-processing, ...) in a way to prevent the backend to be stuck or slow for front-end users and external services.

  • it is possible that the multiplication of backends is also related to security (compartmentalisation and access controls): this architecture shifts quickly data away from the Confidential data backend that interacts with the outside world and is more exposed to attackers.

Additional remarks:

  • IMHO, if you remove the word "confidential", it looks just like an ordinary architecture to handle big volumes, with the "relevance workflow" being some kind of analytic process producing analytical results (poetically called "insight" here) that are stored with some measurement of their expected accuracy.
  • My answer was written before reference of the patent was added to the question. It therefore doesn't take into account additional information provided in the patent claims.
  • "Perhaps the rest of the patent explains more about the rationale." – The patent says that the drawings are merely examples of potential implementations, but do not impose any requirements or limitations. I didn't read it completely, but I guess it would not give a rationale for something that is not really meant to be part of the claim the patent makes anyway. – Jörg W Mittag Dec 8 '18 at 9:23
  • @JörgWMittag Indeed... At the time of my answer, the reference to the patent was not yet in the question. – Christophe Dec 8 '18 at 14:53

Not the answer you're looking for? Browse other questions tagged or ask your own question.