I've got 2 systems over a network. The 2nd system produces some results based on the current contents of system A. How can I efficiently produce results in system B if getting the contents of system A takes a long time?
I've got a system, system A, that keeps customers (personal details, etc). The user can manually enter hits in this system.
I've got another system, system B, that performs, every night, some checks on each customer. It checks if a customer appears on a restricted-persons list producing a report of hits. A user checks on this system each morning and screens manually the produced hits. If a hit is approved, it is then send via POST for saving into system A.
User might have entered a manual hit in System A, which was also found automatically in System B.
GETrequest to system A, every time I present system's B overnight results to the user, to get the existing hits and filter them out of hits produced by System B
In order to ameliorate this mismatch, every time I present the results of system B in the morning, I can send a GET request to get the existing hits of a customer in System A and then perform a comparison, filtering out hits that exist in A and B's hits.
This allows me to show only the new hits in System B. The user screens only the new hits that were found.
A problem with this solution
I can't efficiently know beforehand how many actual new hits exist in System B. I need to
GET 50,000 customers, from System A, each time I want to get the number of new hits.
GETrequest to system A, when a hit is produced by System B, overnight, to see if it's really a new hit that doesn't exist in System A.
Problem with this solution
Since the GET request is sent at the time a hit is produced while checking the list and not when the results are presented to the user there might be a mismatch. The user might have not tended to the results of System B and went ahead and manually entered a hit in System A. System B would still show that manually-entered hit from System A as a new hit in System's B morning results.
Is there a standard way of dealing with such problems?
- The 2 systems are built on Node.js/Express framework. I have build those systems myself so I have full control over them.
- The systems can communicate with each other using a REST API.
- I'd like to keep the 2 systems decoupled, honouring the Single Responsibility Principle - i.e System A just holds it's data, System B performs automatic screenings etc. I'd specifically like to avoid intertwining functionality between each system