our startup is working on an integration for various customers. All of the customers are required to provide their data in one unified way = we chose XML files because of legacy compatibility and ease to validate the files with XSD schemas.
The scenario should be that the customer provides their XML file which we will periodically (about 6 times a day) download and parse the data into our databases.
I am thinking that this could be managed by a single serverless function = simply parse the XML file with up to 1 million lines, compare with already existing records (if there are some updates) and if so, trigger an update event.
I am looking for a design or approach that would be the best match for this. Also, if there is a better way of what I just described, I am all ears for suggestions on how to improve the process.