I'm in the planning phases of trying to build a distributed file processing system in Java and I'm looking for feedback and advice:
Problem : There are a large number of files continuously posted on an FTP server that we need to grab, process and pass on.
Solution Idea: A master node will look for new files on an FTP server, and assign processing work to child nodes. The master node will send a JMS message to the child telling them which file to process and the child will send a response back when it is done, and ask for more work.
If the master node goes down for some reason, one of the child nodes should presume the role of master. My idea for implementing this was to have a "lock" collection in a MongoDB that contains info about the master node, as well as a lock expiration time. Every 15 seconds or so the master node will refresh their lock and update the expiration time to 30 seconds in the future. If the child nodes sees the lock is expired, one of them will assign itself as the master node.
I'm looking for feedback on this design, and wondering if anyone has advice on improvements/java frameworks or tools that already exist that I can leverage for something like this.