1

I've got a legacy application that had a fragile unit test (which is really more of an integration test) that used a lot of Thread.Sleep calls to wait for events to happen.

While trying to introduce some determinism into the tests with some configurable callbacks I came across a test that is attempting to validate that given a certain input, another event doesn't occur.

It is also using Thread.Sleep to wait for 100ms before checking to see if the callback was called, but I'm thinking that a delay in execution could result in a false positive.

I'm trying to keep the test suite snappy without introducing longer delays by increasing the value of the wait.

For the positive tests I've created some code barriers that get resolved through the course of the test, but I don't see a pattern to deterministically determine that an event is not going to occur.

4
  • or mock the executor that runs the task and ensure the task completes? You did use DI for that, didn't you? Apr 17, 2015 at 16:11
  • Unfortunately it is a legacy code base and we are long way from getting DI rolled in. I'm working on incremental improvements as I dig my way through a maze of singletons. Apr 17, 2015 at 16:17
  • Do you have to check the results as you go (IOW after each Thread.Sleep) or can you simply set some flags as events occur and wait until all the steps are complete and then you could have 1 sleep at the end. Then just check the flags to verify if events occurred or not. The total amount of Sleep time is probably going to be less this way, which will allow you to give more time for the problem event to be sure it completes. Otherwise, isn't there a requirement for the max completion time and if the event occurs after this time then that means the test failed, like it is reporting.
    – Dunk
    Apr 17, 2015 at 18:38
  • I believe that there a pauses in the test where the system must wait for background tasks to happen before continuing with the test. It isn't just piecemeal verification on the test side. As I said, not really a unit test. I'll focus on trying to break up what the test is trying to verify into different unit tests that are isolated and deterministic Apr 17, 2015 at 18:46

2 Answers 2

0

Set some variable (accessible to your unit test) when the event occurs. If the event doesn't occur, the variable doesn't get set; you can check for that in your unit test.

5
  • That is essentially what the code does right now. The issue is that the code is set up so that 'at some unknown time in the near future' the variable might be set, so we wait. There's never a guarantee that the variable won't be set just after the check. This question was a long shot. Apr 17, 2015 at 17:30
  • How long do you have to wait before you declare success? Apr 17, 2015 at 17:32
  • Right now the test is set to 250ms, but we have some slower machines that are failing. Not a big deal, but occasionally the shared CI server can run slow and the goal is to ensure that it will always pass. Apr 17, 2015 at 17:35
  • You have to decide what the testing metric is going to be. It's not "at some unknown time in the future," because you can't wait forever. You could perform some sort of White Box testing, but that's not really a unit test anymore, is it? At some point, you're predicting the future. Apr 17, 2015 at 17:38
  • Thanks for your input. That was my thought, I was reaching out to see if there was anything I'd missed. The ultimate goal is to break this test up into actual unit tests and move this to a different integration suite. Apr 17, 2015 at 17:48
0

If the executor is a singleton then you should replace the executor returned by its getInstance() (or stored in the static INSTANCE field) with your mocking framework until you can put DI in place. That way you can monitor the task queue and be know when no more tasks are pending completion.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.