Replacing database polling with messaging queues

Question

We have a background job(java batch) which listens/reads from an MQ and persists a message onto the database(local table). Then a trigger sends this message into a queuing table in WAITING status. My polling program (multithreaded in java) continuous polls for WAITING records, picks them up and sends them to the proper consumer ( java program ). Also messages of same type needs to be processed synchronously. (same order number but both arrive at the Nth min.one needs to complete before another starts) However in this case due to database polling there was a possibility where same record can be picked up by two threads which caused an issue so we put the polling as synchronized which is now resulting in deadlocks.

How can i remove this database polling and make use of an event based mechanism or queues which would be viable rather than database polling and also preserve synchronization.

Edit-- Further message processing is comprised of 4 tasks which are async in nature but all the tasks need to be completed before processing the 2nd message of the same type(orderid). Parent process needs to be synchronous and child are async . Will 2 queues be needed for the same?

Answer 1

I've done this before using Kafka. Have the program that persists each record, post a message onto a second queue after the insert succeeds. Your consumer can then poll for messages on the second queue and do its work. Your MQ should be able to handle multiple consumers reading from the same queue as well as processing in order.

Regarding your edit ... It depends on how your parent consumer communicates to the children. You could stick with 2 queues if all consumer code is in the same JVM. The parent consumer should act as the synchronizer: it would pull off the messages in sequence, then spawn the children tasks as threads to do their asynchronous bit, then once all children are completed, it can pull the next order. Children could communicate their completion status to the parent via callbacks, or the parent does a thread join on the children.

If the parent and children are running in separate JVMs, then you'd have to consider another method of communication. In this case. a 3rd queue could be used (which all children use to communicate their statuses). I've done this also with API callbacks to the parent when the parent has that ability.

Either way, the parent has to wait for all completion statuses before it pulls another order message off the 2nd queue. You should also have a way to timeout and alert you if the children are taking longer than expected.

Answer 2

I would suggest that the program reading from the MQ would dispatch the message directly to the consumer, bypassing the database completely. Eventually, the message can also be inserted into database for persistence and history.

Alternatively, this program can insert the message into the database and send events to the consumer program with the line id to be consumed, preserving synchronous behavior.