5

We are working on a integration between two large systems. System A is a database with employee data, and System B is an external system that is used for the daily contact with the customers (i.e meetings, locationdata,work order,work status, and so on). We will deploy a ESB in the middle.

There are 3 instances of System A, and 1 instance of System B. This is due to legal restrictions and we cannot have all System A data in the same database.

In System B we have 20-30 active contacts each day, where some of the data from System A is used continously.

We need to update meeting, location, and work status in System B throughout the day. And have two proposed solutions for integrating these systems.

Solution A: Publish-Subscribe - System A sends out update messages (This is based on HL7 ADT Standards) and the instance of System B picks up the data that is for them. This is done as broadcast and will generate 1000-10000 messages pr day. It will send information from System A on all employees, thus generating a message for every employee in System A (1,2,3). There is only a subset of employees from System A that is "active" and interesting for updates in System B. But messages will flow on ALL employees since data is updated by other systems aswell.

Soltion B: System B ask for updated information through a request-reply solution. Will generate empty calls, but will only ask for the employees "active" in System B on given day.

Comments on solutions? Pros - Cons? What is the most infrastructure "heavy" solution between these two?

Hope there is someone out there who could give me som pointers :)

Improve this question
4
  • You say in paragraph 2 there are 3 instances of A and 1 of B, then later talk about the 3 instances of system B... can you edit your post to reflect which is correct?
    – James World
    Jun 5, 2013 at 11:33
  • Edited now. My head was spinning... :)
    – kincaid
    Jun 5, 2013 at 14:47
  • @kincaid how does the publish service work? Does it publish the whole employee Data 1000-10000 times a day? That's by an update every ~10 sec. And System B makes a request ~every hour?
    – Rafael Emshoff
    Jun 6, 2013 at 10:23
  • System A will push data on every employee update, but there are triggers on what fields we are looking for. And yes, it will push on all the employees, even the ones not active in System B. We have done some measurements and we are talking about 3000-5000 updates every day. And on any given day, System B is interested in about 250 of these updates.
    – kincaid
    Jun 7, 2013 at 7:22

3 Answers 3

Reset to default
5

Solution B only works if you request the data from System A every time it is required by System B. Otherwise you cannot guarantee the information is up to date. This is not as bad as it sounds and is in fact the basis of all SOA architectures.

The decision of whether to push (Solution A) or pull (Solution B) depends on the ratio of updates in system A to the number of queries in System B.

Where you have lots of updates in system A versus a few queries in System B then "pull" is the obvious choice.

Where you have relatively few updates in system A and lots of queries in System B then "push" is the way to go.

Where you have lots of updates and lots of queries either choice is equally good (or bad!).

Improve this answer
2
  • 1
    Yes, this is the discussion we have had the last couple of days :) There is no exact answer, and we have to do some measurements to see what will be the cheapest regarding resources.
    – kincaid
    Jun 7, 2013 at 7:29
  • I have accepted this answer, but all answers are actually good solutions depending on what "world" we are deploying in. We might have to change our strategy as the map changes.
    – kincaid
    Jun 10, 2013 at 18:50
2

Since you are using System B in real-time, you should chose whatever architecture you can technically implement, that will give you the most up-to-date data in System B, whilst using the least resources.

You need to look at the implementation details of Solution A and B, and see what gives a better current-data to resource(memory, CPU) consumption. Ideally you'd have a publish service on System A per record, that informs System B when an individual record in System A has been changed... but that is neither of your proposed solutions.

Improve this answer
2
  • 1
    You beat me to this answer. In other words, what's wrong with using the observer pattern.
    – Dunk
    Jun 6, 2013 at 14:40
  • We have discussed the observer pattern, and the solution you proposed. System A has a publish-per-update (based on field triggers) . The problem is that System A does not know what employees are active on any given day in System B. If we could add that information to an observer-pattern, then we would only get pushdata for the employees in that list of active employees.
    – kincaid
    Jun 7, 2013 at 7:34
1

If updates only flow from A to B, then B could just cache data from A. B should only read data from A when it needs to and no need for A to send data (publish) without being asked. Then you have to manage what to do with stale cache (A has been updated since last time B updated). But what if B only needs data when A has been updated? B can periodically ask A if there's any update since B last check, and only get updated entities.

R. Fielding gave comment about why pub/sub isn't scalable here: http://roy.gbiv.com/untangled/2008/paper-tigers-and-hidden-dragons which where I get my answer from.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.