Am I looking at multithreading the wrong way? (Java)

Question

This is a re-post from my question in StackOverflow, so here goes:

For the past few weeks now I've been studying Concurrency(Multithreading) in Java. I find it difficult and rather different than anything I've encountered in the Java language so far(or in programming in general). Often I have to reread and reread over and over again until I start to understand a small concept fully.

It's frustrating and I've wondered why this part of the Java programming language has given me so much trouble.

Usually when I look at the code of a single-threaded program I look at the main method and start going step by step in my mind through the whole execution(like a debugger). Throughout this process I try to keep in mind EVERYTHING like variables and their states(values) at every point in the execution. Often times when doing that I even stop at certain points and think how the program execution would alter in different scenarios. If I can go through a program from start to finish like that, I feel like I've fully understood the code and the material.

The problem that I have, I suppose, is that when I try to apply this method for a concurrent application, there are so much things happening at once(sleep(), synchronized methods, acquiring intrinsic locks, guarded blocks using wait(), etc.) and there's so much uncertainty of when something will execute, that it becomes nearly impossible for me to keep up with everything. That's what frustrates me, because I want to have a feeling of "I have control over what's happening", but with concurrency that's impossible.

Any help would be appreciated!!!

Answer 1

A concurrent system is inherently more complex than a single-threaded system. In fact, complexity scales exponentially with the number of threads: If I have three threads that can be in any of 5 states each, my total program has 5^3 = 125 states. We have to take care to limit the cognitive load.

The central theme is to limit interaction between threads. If data is shared between threads, that data should not be modified (only read from). Clearly define how, where, and when threads communicate. If multiple threads require read-write access to some resource, guard that resource by forcing all calls to go through synchronized methods.

There are a couple of patterns that help you to use threads sensibly. For example, you might have a thread pool where each worker thread retrieves units of work from a task queue. This allows a main thread to offload expensive tasks to the workers. This is particularly suitable for CPU-intensive problems that are easy to parallelize.

A variation of this is an event loop. The event loop processes events from a queue and dispatches tasks and updates to other threads, without doing relevant work itself. In a model-view-controller architecture, this would mediate between the view and the controller, and between the model and the view. The point is that all communication goes through the event loop, so all state changes are only triggered at this single location.

There are also cases where multithreading is not a suitable mechanism. If you want to use threads because you want to do some work while you are waiting for some result, using asynchronous/non-blocking operations is usually easier. They might still use threads under the hood, but you are shielded from the complexity of threads. And unless you want to create a user-visible pause, never sleep() – instead, use semaphores, wait/notify, or thread barriers to make sure threads are synced up.

Answer 2

I want to add one thing. Although the existing answers mention this they do not expand on it: what’s difficult in concurrency is not just the interactions between threads (wait, notify, locks, etc) but the shared state.

Most applications have state. As you use the application you pass from one state to the other. The state is represented by objects that you create, mutate, and delete. When you manage the state from within a single threaded application it's more or less easy to think about state. But when you introduce multiple threads on the application you now get shared data which is not as easy to handle.

To give a simple analogy of what shared data means, imagine you have 10 cooks with 10 kitchens. If you ask them all to cook different dishes, each cook with his own kitchen can make it happen with no issues. Now take those 10 cooks and make them use the same kitchen at the same time, trying all to prepare their dishes at the same time. What will happen? I’ll let you imagine :). Pretty much that’s what happens too when you have one piece of data and multiple threads.

Because thread execution can interleave, with the threads trying to mutate the same piece of information, you can end up with all sorts of issues like:

• corrupted data (an i++ might all of a sudden increment the variable twice);
• stale data (because of CPU core caches some threads might not even see the i++ increment);
• crashes of your application (the application entering an illegal state because some data somewhere got messed up);
• etc.

Some solutions to this problem of shared data include:

• synchronizing access to that information so that threads take turns when mutating the shared data. This can cause all sorts of issues like deadlocks or having parallel executing work perform way worse than if you would do the same work sequentially, etc;
• Using immutable data (if data doesn’t change then no need to protect access to it);
• Using message passing, like in the Actor model (you don’t actually have shared data across the entire application, each actor has its own data);
• Using a pure functional language (try to remove shared state all together).
• etc

So the way to look at this is to see threads as individual processes that work on their own data but also with data from the outside. If it’s their own data it’s simple, it’s like execution in a single threaded environment. When the data is not their own you have to carefully look at what’s being shared and reason about it to make sure that the data remains in a valid state no matter what.

It’s not something easy, you have to read a lot about it so you know what you can be exposed to (deadlocks, atomicity, race conditions, starvation, livelock, liveness, contention, etc). But even after you understand a lot and start to manage all of this, it still won’t be easy. But it’s something you need to learn as things are moving to a distributed world (be it on different servers or different cores of the same CPU chip).

Answer 3

To complement @amon's most excellent answer and take a step back to look at the bigger picture.

Divide and Conquer

Programming is all about dividing a problem in smaller manageable chunks. You divide the problems in data elements, in actions that can manipulate the data, in means to display this data.

These in turn can implement interfaces to bring changes in behaviour without changing the overall structure.

The better you separate the code, the more isolated each task is from the other task the easier the code is to maintain and grow.

The same principle applies to threading. You need to carefully control the interface of the thread.

Take a method in a class as example, specifically the one that perform the hard work :

You have basically two ways to create it, either use the class's fields directly with a parameterless signature returning void or give the function all it needs through arguments and have it return a value. Sure in the latter case you would need another method that would mutate the class's fields but this way you limit what can read and write the fields. That function, now isolated from the rest through a very clear interface becomes very easy to put in it's own thread. You no longer need to keep in mind all of the function's internal interaction points. The function's signature itself defines all the interaction points. Nothing hidden, complete transparency.

Behind SOLID

You say that you can keep everything in mind and basically run the application in your head. That is a very nice skill to have, but what it also tells me is that you probably never tackled a system that was very big. Should you encounter one I would expect you would run in the same issues as you are now with threads, that is, until you really sink in the core principles at the heart of OOP (at the heart of programming really). I say all this with much humility, as I too often struggle with complexities such as these but I find that very often these arise not so much because the problem is inherently complex but more because it was not modeled optimally. Too many interaction points, or coupling if you prefer, too many side effects, too much abstraction leakeage, intermingled concepts (class or function doing more than one thing) and so forth. Taking a step back, rethinking the structure, dividing the problem further to better conquer it.

Threading is hard because it tends to expose these issues with your program's structure. Many things that were "good enough" before now become unmanageable.

My 2 cent from my own humble experience, hope this helps.

Answer 4

I want to have a feeling of "I have control over what's happening", but with concurrency that's impossible

And that will likely hold true even after many years of experience with multi-threaded applications. Most important thing is to learn for your specific programming languages what variables can change when, and how access to such variables can be synchronized. That will help you to design your code such that concurrency issues won't occur in too many places, but of course you still won't be able to fully avoid them.

As for the dead-locking problem which you might run into soon, look at the Coffman criteria.

Answer 5

You have to let go. The complete system with everything that is going on is much too complicated to keep track of. And it's not needed at all.

You write one small portion, and all you have to make sure is that if everyone else does their part correctly, then your part will work.

I'll give you an example: I have a method that reads some data from a server, and calls some callback when the data is available or the read fails. Now this call goes off and checks whether you are logged in to the server, shows the UI for logging you in if needed, where you either login or not, fails if you don't login, then eventually goes to the server, might run into the problem that your phone's WiFi is disabled, asks the user to turn their WiFi on, switches to the Settings application to be helpful (now we're not just in another thread, but in another application), eventually comes back, handles other errors that it can handle, and that's only a quarter of what it does, and eventually it calls your callback with the data or an error. And guess what: All that stuff going on in between is of no importance whatsoever. You just ignore it.

Answer 6

Seems like you understand your system most intuitively through control flow. I tend to be similar.

If you're like me in this regard, you might exhibit a strong distaste not only for complex multithreaded code with lots of shared state but also deep call stacks and event-driven programming. Event-driven programming can be almost as difficult to reason about in terms of control flow as multithreading. A user might push a button which resizes a pane which triggers a resize event which triggers more resize events on child widgets, of which one is a viewport which triggers a GPU render event picked up by a rendering thread which then renders the scene and triggers a finished event, at which point the viewport picks it up and triggers a repaint event, and so on.

I hate working in such systems. As a result I've developed a strong preference towards what I call "flatter" code as opposed to "deeper" code, with shallow call stacks and fewer threads running at once, with meatier functions where you can look at a procedure and understand a good chunk of how your program works from it as opposed to a complex graph of teeny functions and objects interacting with each other where you have to connect the tiniest puzzle pieces together to see the big picture. Entity-component systems made a wonderful fit for me personally, since in those only the systems contain any functionality, and all of them contain meaty functionality that loops through a bunch of entities and does things with them (an entire PhysicsSystem, e.g.). That makes it easy for me to comprehend the big picture.

Now the usual way you're supposed to do it is to not be concerned with such details. You can test each individual unit thoroughly, reason about its correctness and thread safety, and supposedly work that way from bottom-up and compose an enormous codebase and not feel lost when you look at the system top-down. That doesn't really work for me so well. That's not to say that the system didn't work. I've worked with reasonably well-written codebases that had sprawling and cascading events with lots of asynchronous code that worked quite well with each little unit doing what it's supposed to do. Yet I never felt comfortable in spite of that because I couldn't break off large chunks of the system and reason about them so well without worrying about details, and most importantly, I couldn't confidently predict what would happen if we changed something with such a busy ecosystem consisting of countless interactions between teeny things.

You can unit test microwaves and find that they do exactly what they need to do in all scenarios and heat things up. You can test plastic wrap and see that it works perfectly fine in all cases for wrapping food. But try putting plastic-wrapped food in the microwave... whoops! There are system invariants to maintain that go way beyond the correctness of a granular object or function. As an overseer type over a codebase, I always cared more about the broad invariants than the small ones, since it's easy enough to maintain granular invariants. I suppose you could try to model bigger and bigger objects that compose higher and higher-level concepts and maintain broader and broader invariants until your unit test is effectively an integration test for the entire application, but I've never seen such coarse-level testing applied thoroughly, ever, or else the testing would almost guarantee that the entire software is free of bugs.

So I find it's worth taking it easy on the cascading events and asynchronous tasks and persistent threads. You don't have to concurrently run a PhysicsSystem at the same time as the RenderSystem. They both have bulky enough work to do to, say, just parallelize the loops they perform while calling them in a sequential order and potentially get even better frame rates, focusing on making those systems finish faster rather than run simultaneously, and parallel loops are much easier to reason about than having entire systems running in parallel. I find that so much easier to reason about.

But anyway, for you, I'd try to find indivisible chunks of the system you can break off that are reasonably high-level, like OfficeBuilding, not the little things like the plastic wrap and the microwave inside taken individually, where you can reason about this broad chunk of your system's correctness in spite of having its code being executed in parallel. That unit has to be thought as an independent unit and kind of like a black box so that you can then reason about your system as a whole as a small collection of huge black boxes, all of which are guaranteed to be thread safe, without understanding all the details. ECS lets you do this in a "flat" way which extends in an orthogonal fashion, OOP will tend to favor a "deeply nested" way of big behavioral objects encapsulating medium behavioral objects which encapsulate smaller behavioral objects and so on.