-4

A while ago Herb Sutter wrote The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software which I basically interpret to mean that, in order to improve performance, software engineers are going to have to embrace concurrent, specifically multi-threaded programs.

When I first read the article I thought I would have to improve my understanding of threading, but now there is async code in many modern languages.

Is it far to think that the native implementations of async code make embracing concurrency or multi-threading easier in that we just have to use the async code provided by languages to take advantage of the multiple cores on a chip?

4
  • 1
    But the native implementations of async code in Python and C# don't take advantage of multiple cores, do they? Jun 25, 2021 at 21:34
  • not sure if they do in python and C#, but they definitely do in some languages
    – Jack
    Jun 25, 2021 at 22:05
  • 2
    Solving the "free lunch is over" problem would mean the programmer could just rely on improved hardware for getting more performance, without taking any special care for it in a program. But async/await has to placed deliberatly, and to utilize more CPU cores, programmers will still have to take care a lot of about things like deadlocks and race conditions, there is nothing "free".
    – Doc Brown
    Jun 25, 2021 at 22:49
  • 1
    Asynchrony and threading are somewhat orthogonal concepts. In C# at least, the async and await keywords themselves do not cause any kind of threading or parallelism, though they may sometimes be used to orchestrate multi-threaded code. However, async/await in C# are also very useful in purely single-threaded applications which use asynchronous operations such as HTTP requests or database commands. There are a lot of excellent articles about the keywords on Stephen Cleary's blog - blog.stephencleary.com/2012/02/async-and-await.html Jun 25, 2021 at 23:44

1 Answer 1

1

Async/await is about waiting more efficiently, not really about increasing CPU performance. This happens to allow you to utilize your cores more efficiently, but is more about efficiently doing I/O: while one task is waiting for an IO operation to complete, another pending task can run. This has little to do with Sutter's 2005 article about the limits of single-core systems. Async/await is not directly related to multithreading.

There is concurrency and parallelism. Parallelism is about computations happening at the same time. Concurrency is about computations not happening sequentially. While parallelism is a kind of concurrency, another concurrent model is that computations are interleaved.

Async/await allows for such efficient interleaving: an executor or event loop keeps track of completed IO tasks and other events, and schedules dependent tasks to run appropriately. This can happen without the overhead of threads, in particular without having to involve the operating system (which is comparatively slow). Without such an executor, threads would spend most of their time waiting for an event to happen. This is especially relevant for server software: traditionally, a server would have a pool of worker threads, e.g. 2× the number of cores. This limits the number of tasks that can be “in flight” at any time. With an async executor I can have exactly as many threads as cores, but they can juggle as many async tasks as fit into memory. Thus, a single web server can juggle many thousands concurrent (but not parallel!) requests.

Python is a bit special, because CPython's global interpreter lock (GIL) prevents true parallelism. Here, async/await helps to pack more IO-bound tasks into the single Python thread that can currently run. The only ways for CPU-bound Python programs to make use of multiple cores are to either call into C code that manages its own threads (e.g. with OpenMP), or to run multiple Python processes. Before Python introduced async/await syntax, it could be emulated with yield (generators), which is also a kind of interleaving computations on a single thread.

The real advantage of language-level async/await is that it makes it easier to think about concurrent code. When a function awaits, it is clear that a different task may run at that point. But in between awaits, it behaves like traditional, non-concurrent code. Without async/await we would have to use callbacks or event listeners, which is much more difficult to structure well. Indirectly, this also helps to write better parallel software. But the async model is not about parallelization, but about interleaving separate computations efficiently – not about going faster, but about doing more stuff with the same resources.

Not the answer you're looking for? Browse other questions tagged or ask your own question.