7

The question is does it make sense to write a unit test which asserts that the current thread is the main thread? Pros/cons?

Recently I've seen the unit test which asserts the current thread for the callback of service. I'm not sure, that it's a good idea, I believe it's more kind of integration test. In my opinion, the unit test should assert method in isolation and should not know about the nature of the consumer of service.

For example, in the iOS, the consumer of this service is intended to be a UI which by default has a constraint to run a code at the main thread.

  • 1
    FWIW, in our code, rather than making it a unit test assertion, we make it a runtime assertion, since that's where it really matters. – user1118321 Dec 15 '18 at 5:44
  • Seems more like an assertion rather than an actual test case. – whatsisname Dec 15 '18 at 21:14
9

Let me show you my favorite unit test principles:

A test is not a unit test if:

  1. It talks to the database
  2. It communicates across the network
  3. It touches the file system
  4. It can't run at the same time as any of your other unit tests
  5. You have to do special things to your environment (such as editing config files) to run it.

A Set of Unit Testing Rules - Michael Feathers

If your unit test insists that it be the "main thread" it's hard not to run afoul of number 4.

This may seem harsh but remember that a unit test is just one of many different kinds of tests.

enter image description here

You're free to violate these principles. But when you do, don't call your test a unit test. Don't put it with the real unit tests and slow them down.

  • Good point and great link to the article, thanks! – Vladimir Kaltyrin Dec 14 '18 at 17:17
  • How do you unit test a logger? Do you have to moc IO? That seems obnoxious. – opa Dec 14 '18 at 21:47
  • 1
    @opa if your logger has interesting business logic you isolate that and unit test it. Boring code that confirms we have a file system doesn't need unit testing. – candied_orange Dec 14 '18 at 22:36
  • @candied_orange So now you have to expose internal logic? Because how else are you going to separate the string generated by the logger before it is output to a file with separating it out into a private method? – opa Dec 14 '18 at 22:54
  • @opa If you need to test that you can do it by passing in your own stream. Unit tests test units not system resources. – candied_orange Dec 14 '18 at 23:19
4

Should you write a unit test which tests if a thread is the main thread? Depends, but probably not a good idea.

If the point of a unit test is meant to verify proper functioning of a method, then testing this aspect is almost certainly not testing this aspect. As you said yourself, it is conceptually more of an integration test, since the proper functioning of a method is not likely to change based on which thread is running it, and the one responsible for determining which thread gets used is the caller and not the method itself.

However, this is why I say it depends, because it may very well depend on the method itself, should the method spawn new threads. Though in all likelihood this isn't your case.

My advice would be to leave this check out of the unit test and port it to a proper integration test for this aspect.

2

should not know about the nature of the consumer of service.

When a consumer consumes a service there is a "contract" express or implied about what functionality is provided and how. Restrictions on what thread the callback can happen in are part of that contract.

I presume that your code runs in a context where there is some sort of "event engine" that runs in the "main thread" and a way for other threads to request that the event engine schedules code is run in the main thread.

In the purest view of unit tests you would mock absoloutely every dependency. In the real world very few people do that. The code under test is allowed to use the real versions of at least basic platform features.

So the real question IMO is do you consider the event engine and the runtime threading support to be part of the "basic platform features" that unit tests are allowed to depend on or not? If you do then it makes perfect sense to check if a callback happens on the "main thread". If you are mocking the event engine and threading system then you would design your mocks so they can determine if the callback would have happened on the main thread. If you are mocking the event engine but not the runtime threading support you would want to test if the callback happens on the thread where your mock event engine runs.

1

I can see why it would be an attractive test. But what would it actually be testing? say we have the function

showMessage(string m)
{
    new DialogueBox(m).Show(); //will fail if not called from UI thread
}

Now I can test this and run it in a non UI thread, proving that an error is thrown. But really for that test to have any value the function needs to have some specific behaviour that you want it to do when run in a non UI thread.

showMessage(string m)
{
    try {
        new DialogueBox(m).Show(); //will fail if not called from UI thread
    }
    catch {
        Logger.Log(m); //alternative display method
    }
}

Now I have something to test, but its unclear that this kind of alternative would be useful in a real life

1

If the callback is documented to not be thread-safe to invoke on any other than, say, the UI thread or main thread, then it seems perfectly reasonable in a unit test of a code module which causes the invocation of this callback to make sure it isn't doing something that is not thread-safe by working outside the thread that the OS or application framework is documented to guarantee safe.

1

Arrange
Act
Assert

Should a unit test assert that it’s on a specific thread? No, because it’s not testing a unit in that case, it’s not even an integration test, as it doesn’t say anything about the configuration taking place in the unit...

Asserting which thread the test is on, would be like Asserting the name of your test server, or better yet the name of the test method.

Now, it might make sense to Arrange for the test to be run on a specific thread, but only when you want to be sure that it will return a specific result based on the thread.

1

If you have a function that is documented to only run correctly on the main thread, and that function is called on a background thread, that's not the functions fault. The fault is the caller's fault, so unit tests are pointless.

If the contract is changed so that the function will run correctly on the main thread, and report a failure when run on a background thread, you can write a unit test for that, where you call it once on the main thread and then on a background thread and check that it behaves as documented. So now you have a unit test, but not a unit test checking that it is run on the main thread.

I would very much suggest that the first line of your function should assert that it runs on the main thread.

1

Unit testing is only useful if it's testing correct implementation of said function, not correct usage, because a unit test can't say that a function is not going to be called from another thread in the rest of your codebase, just as it can't say that functions won't be called with parameters that violate their preconditions (and technically what you're trying to test is basically a violation of a precondition in usage, which is something you can't effectively test against because the test can't restrict how other places in the codebase use such functions; you can test if violations of preconditions result in approriate errors/exceptions, however).

This is an assertion case to me in the implementation of the relevant functions themselves as some others have pointed out, or even more sophisticated is to make sure the functions are thread-safe (though this is not always practical when working with some APIs).

Also just a side note but "main thread" != "UI thread" in all cases. Many GUI APIs are not thread-safe (and making a thread-safe GUI kit is damned hard), but that doesn't mean you have to invoke them from the same thread as the one which has the entry point for your application. That might be useful even in implementing your assertions inside the relevant UI functions to distinguish "UI thread" from "main thread", like capturing the current thread ID when a window is created to compare against instead of from the application main entry point (that at least reduces the amount of assumptions/usage restrictions the implementation is applying to only what is truly relevant).

Thread safety was actually the "gotcha" tripping point in a former team of mine, and in our particular case I would have labelled it the most counter-productive "micro-optimization" of them all of a kind that incurred more maintenance costs than even handwritten assembly. We had rather comprehensive code coverage in our unit tests, along with rather sophisticated integration tests, only to encounter deadlocks and race conditions in the software that eluded our tests. And that was because the developers haphazardly multithreaded code without being aware of every single side effect that could possibly occur in the chain of functions calls that would result from their own, with a rather naive idea that they could fix such bugs in hindsight by just throwing locks around left and right, and perhaps even getting a false sense of confidence from their test coverage*.

I was skewed in the opposite direction as an old school type that distrusted multithreading, was a real latecomer to embracing it, and thought correctness beats performance to the point of rarely ever getting use out of all these cores we have now, until I discovered things like pure functions and immutable designs and persistent data structures which finally allowed me to fully utilize that hardware without a worry in the world about race conditions and deadlocks. I must admit that all the way up until 2010 or so, I hated multithreading with a passion except for a few parallel loops here and there in areas that are trivial to reason about thread-safety, and favored much more sequential code for the design of products given my grief with multithreading in former teams.

To me that way of multithreading first and fix bugs later is a terrible strategy to multithreading to the point of almost making me hate multithreading initially; you either make sure your designs are rock-solid thread-safe and that their implementations only use functions with similar guarantees (ex: pure functions), or you avoid multithreading. That might come across a bit dogmatic but it beats discovering (or worse, not discovering) difficult-to-reproduce issues in hindsight which elude tests. There's no point optimizing a rocket engine if that's going to result in making it prone to unexpectedly explode out of the blue halfway along its journey to out of space.

If you inevitably have to work with code which is not thread safe, then I don't see that as an issue to solve with unit/integration testing so much. Ideal would be to restrict access. If your GUI code is decoupled from business logic, then you might be able to enforce a design which restricts access to such calls from anything other than the thread/object which creates it*. That's far mode ideal to me is to make it impossible for other threads to call those functions than to try to make sure they don't.

  • Yes, I realize that there's always ways around whatever design restrictions you enforce typically where the compiler can't protect you. I'm just speaking practically; if you can abstract "GUI Thread" object or whatever, then it might be the only one handed a parameter to the GUI objects/functions, and you might be able to restrict that object from having access to other threads. Of course it might be able to bypass and dig deep and work its way around such hoops to pass said GUI functions/objects to other threads to invoke, but at least there's a barrier there, and you can call anyone who does that an "idiot", and not be in the wrong, at least, for clearly bypassing and seeking loopholes for what the design obviously was attempting to restrict. :-D That's actually a very practical litmus test to me is like how confidently you can call someone an "idiot" in misusing a design, without the risk of you actually being the idiot for designing something prone to misuse.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.