I am working on designing a brand new Data Ingestion Pipeline with the Key highlights of the new project are as follows:
- Download and Update data to/from SharePoint using SharePoint APIs
- Download and Update data to/from JIRA/incident management application using JIRA APIs
- Download and Update data to/from SQL sources using provided APIs
- Download and Update data to/from external custom applications using APIs
I am considering the micro-services architecture for the above project where I will be looking at creating 4 separate services for each of the above purpose.
And finally, a batch processing client that would execute all these API services using C# .NET
But I have been wondering if implementing micro-services architecture will be overkill, and rather all I really need is a single client calling all these APIs directly without having to create above individual services.
And additionally, regarding setting up the project in Visual Studio, should all these services be in their own separate solutions or rather be a part of one .NET solution with multiple projects in it.