I am developing a web application for a client, connected to 4 differents sources :
- E-commerce website
- CRM
- Emailing platform
- Surveys platform
All the sources have REST Api's.
The web application has to get the data from the sources and provide a list of customers. Also the client can create/edit customers, so the web application have to send the new / updated customer to the sources.
On the sources, the client can also create/edit customers.
The 4 sources have the same columns (email, firstname, lastname, etc.)
The web application has to provide a unique list of customers (unique by email address).
On the web application and the source i have access to an "updatedAt" value.
This is the workflow i want to setup :
- A crontab each 5 minutes > get last customers from source 1
- Check on the web application if a more recent version exists
- If not insert / update customer
- Then send to the other sources the new / updated customer (the updatedAt value will change on each source)
But on the next cron execution (for source 2) the previous updated customers will be returned because of the updatedAt column, so i'm afraid, the last customers list will increase and increase..
I can't control the change of the updatedAt column on the sources.
I feel stuck!
My questions are :
- What is the best way to handle this kind of system ?
- Is my workflow idea in a good direction or totally wrong ?
Thanks in advance :)