I have been asked to come up with a design for streaming data to the cloud. I have a closed system on a local server. I need to stream recently registered customers or updated customer records on the local DB to a cloud-based CRM and make data available to serve the firm's website. I came up with following design flow:

  • executable to access the database on the local server and post records to the cloud server. This runs every hour. (1)
  • A cloud server on which an api is hosted (2). A client (exe) calls the CRM Api to post records (3)
  • Website gets(4) via api call.
  • Data for website viewing is different from the data posted to CRM

Any suggestions or feedback/criticisms on the design with regards to connectivity etc.. is there a better way to streaming data from closed systems.enter image description here

  • The question whether a 1 hour lag is acceptable isn't an engineering question, it's a product question. It's fine if it meets the product requirements. Commented Oct 31, 2017 at 6:19
  • 1
    Also, what's keeping you from running the synchronization more often if you feel it's necessary?
    – doubleYou
    Commented Oct 31, 2017 at 6:54
  • @AvnerShahar-Kashtan It is a requirement part of the engineering solution.
    – Mo Ali
    Commented Oct 31, 2017 at 11:00
  • any feedback on the design, any criticisms are welcome
    – Mo Ali
    Commented Oct 31, 2017 at 11:01
  • Yes, and so it's not an answer you can get from other people not familiar with the project and its requirements. If this is data that changes every other day, a 1-hour lag is meaningless. If the users check the web site every week, it's meaningless. But we don't know that. Commented Oct 31, 2017 at 11:01

1 Answer 1


It looks fine, but to evaluate it properly I would want to know more about the details of the problem you are trying to solve instead of just the engineering solution that seems to have been proposed.

Why do you need to support entering data in to a local database just to push it out to the cloud based CRM (SFDC?).

Can you not just get people to enter their data in to that directly?

If it is a popular cloud based CRM there are existing products that will keep your local database and the cloud based CRM in sync so you wouldn't need to write any custom code (SFDC data model is quite complex to code around yourself).

Does the website already exist and you are extending it? The cloud based CRM presumably has a web front end people could use.

  • Kudos to you for saying that there are existing products for local db and crm sync. I would like to know more please, what are these products.
    – Mo Ali
    Commented Oct 31, 2017 at 13:14
  • "Why do you need to support entering data in to a local database just to push it out to the cloud based CRM (SFDC?)." - because I need to persist the data to serve the exisitng website's viewing area for members only. I will be building it. Yes, the CRM has a frontend for users
    – Mo Ali
    Commented Oct 31, 2017 at 13:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.