Let me give a bit of background on the problem.

We have a java application which has MQ integration and an Integration Test pack which is also written in java.

The test pack runs all scenarios (positive and negative) by running the test cases (methods) and changing the parameters depending on the scenario. The way it does the assertions is to put a message on the input queues of the application and expects a message in the output queues of the application. The message body is checked because it differs depending on the test scenario.

There is only 1 input queue and 1 output queue in the main application. The number of test cases (methods) are about 300 and due to running all the known scenarios, the number of test cases actually executed are about 2500. Due to this, the test pack takes around 6 hours to execute fully.

Since there is only 1 queue on which the test pack asserts the output message, we have to clear the queue each time before we run the next test case. Otherwise it may lead to mixing of the output messages. For e.g. method call#2 might end up consuming output message for method call#1. Hence we can't run the test cases in parrallel too.

I want to improve this process and would be highly grateful for any suggestions to optimize the test pack. Right now I am thinking to separate out the input and output queues of the main application as per business function and thus assert on multiple queues in parallel. But this approach means I have to change the original architecture so that my Test cases run faster and this is not easy since there are already clients using this application and interacting using the 2 queues and they will have to change their applications too.

Apologies for the long post but I would love to hear inputs of so many creative people in SO community.

Thanks!

  • 1
    Can you spin up multiple instances of the application and queues on separate (virtual) machines? Then you could run tests in parallel. – Bart van Ingen Schenau Jun 10 at 18:15
  • Thanks, that's a valid suggestion. But it would require changes in the infrastructure to spin up so many instances of the application, which would not be easy from a finance/approval perspective of our organization. – phoenixSid Jun 10 at 18:27

One of the things that used to drive me nuts was when eclipse would run my tests in parallel when I hadn't expected it. This vexed me because I'm an old school tester who still thinks writing to the console is useful. Two tests would write to the console and sometimes everything was fine. Sometimes one's text would be mixed halfway into the others.

Then I realized how simple the solution was. Just let each test build it's own string and output that one string at one time. Once I did that I could let the tests run in parallel just fine.

If you don't have sequential dependencies, shared memory, or multiple outputs from each job to keep in order I really don't see why you can't let them run in parallel.

  • There are no sequential dependencies, but there is only one output assertion queue where all tests will send the messages. If the tests are running in parallel, test#3 might read messages from test #2 etc. – phoenixSid Jun 11 at 4:23
  • You just described a sequential dependency. – candied_orange Jun 11 at 4:26
  • no, I thought you were referring to sequential dependencies in the test methods itself for e.g. test#3 must always run after test#2. There is no such dependency. – phoenixSid Jun 11 at 5:19

Since there is only 1 queue on which the test pack asserts the output message, we have to clear the queue each time before we run the next test case. Otherwise it may lead to mixing of the output messages. For e.g. method call#2 might end up consuming output message for method call#1. Hence we can't run the test cases in parallel too.

Assign unique ids to the messages and listen for message with expected id in output queue with an upper time limit. This should get you testing parallel executions.

You can wrap this publish/listen logic in a generic request/response style implementation. From there on, use something like RxJava to compose scenarios of multiple such calls.

  • we already have unique ids. But the problem is, once a message is picked up from a queue it is gone. So for e.g. if test#3 consumes a message and realises it was test#2 message, nothing can be done now. test#2 will never get the message. – phoenixSid Jun 11 at 5:18
  • 1
    So stop consuming messages that aren't yours. – candied_orange Jun 12 at 0:27

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.