I want to build a middleware system that my company can use and expand on to handle the integration points between our various bespoke and 3rd party systems as well as our hosted websites.
I am having some trouble with the design/architecture and would like some advice (please let me know if there is a better site to ask these questions).
The middleware will basically be the broker between our ERP and our other systems such as our website/3rd party retailers etc. It will be mainly dealing with API calls and XML file exports.
I was thinking of:
- Using RabbitMQ as a queuing mechanism.
- Using Quartz.NET as a scheduler contained within a Windows Service. It would be very lightweight and would simply load up a list of jobs from a database, dropping these into the queue when their schedule is reached.
- Using a windows service (I'd call this the agent) which would be a consumer of the RabbitMQ queue. It would monitor the queue and process the jobs as they would come in. Ideally, I would want to be able to run separate jobs at the same time so that one job doesn't necessarily tie up another job from running. We can always scale this portion out if required.
Does anyone see any issue with this approach? I can't find a lot of example implementations on the internet about using RabbitMQ as a worker service in .NET (using multiple threads) and am not sure if using a scheduler service to simply drop items on the queue is the right approach.