The meaning of asynchronous vs synchronous [closed]

Question

What is the meaning of the words asynchronous and synchronous in computer science?

If you google the meaning of the words you will get the following:

• Asynchronous: not existing or occurring at the same time.

• Synchronous: existing or occurring at the same time.

But it seems like they are used to convey the opposite meaning in programming or computer science:

HTML async attribute means that the script will be executed as soon as it is downloaded even if the HTML is still parsing or downloading, which means both processes, the script and the HTML, exist and occur at the same time to me.

Are these terms used to convey the opposite meaning in computer science or am I missing the point?

Answer 1

I would like to give you an answer which is directly related to those definitions you found. When one task T1 starts a second task T2, it can happen in the following manner:

Synchronous: existing or occurring at the same time.

So T2 is guaranteed to be started and executed inside the time slice of T1. T1 "waits" for the ending of T2 and can continue processing afterwards. In this sense, T1 and T2 occur "at the same time" (not "in parallel", but in a contiguous time interval).

Asynchronous: not existing or occurring at the same time.

So the execution time of T2 is now unrelated to T1. It may be executed in parallel, it may happen one second, one minute or several hours later, and T2 may still run when T1 has ended (so to process a result of T2, a new task T3 may be required). In this sense, T1 and T2 are not "occuring at the same time (interval)".

Of course, I agree, the literal definitions appear to be ambiguous when seeing that asynchronous operations nowadays are often used for creating parallel executions.

Answer 2

I find that the best way to understand it is the following:

• Synchronous: We know when it will happen (it happens when this other code ends).
• Asynchronous: We do not know when it will happen.

Note: while we can schedule code to be executed a given clock time, in practice we do not know when it will happen, because it can be delayed – even ignoring messing with the system clock – because the system is busy doing something else. Besides, even if we had the guarantee that it will happen exactly at a given clock time, we are not sure where the execution of our program would be at that time. So, no, code scheduled for clock time are not synchronous.

---

Please notice that in software development we will say, for example, that a task is asynchronous as something about the task in isolation. However, if you want to define it in terms of occurring at the same time or not requires to have at least something else to compare the task to.

Many platforms can do both parallelism and task switching, some have limited parallelism, some cannot do parallelism at all and rely only on task switching... Furthermore, some platforms are unable to interrupt a task and must have them complete before executing a different one... Asynchronous task are an abstraction over all that, such that the system can decide how to run the tasks for the given platform without the developer worrying about it (too much).

It is also worth noting that we can conceptualize, and usually we abstract, getting external input as an asynchronous task. For example: getting text from user input. We do not know when the user will type. This also applies to reading from permanent storage, getting data over the network, or any other external system.

---

By the way, despite some things being fundamentally asynchronous, we can usually pretend they aren't. We do that by having the software block the current execution – and doing nothing else – until it is completed. That is, we can take something asynchronous and wrap it in a synchronous API.

An asynchronous API will allow you to continue execution despite the requested operation not being complete, a synchronous one does not. And from there you get the idea asynchronous – in software – means occurring at the same time (concurrent).

It is worth noting that asynchronous does not imply concurrent. In some cases, the platform can only execute the asynchronous task after the current task has completed. That would be sequential (although the order of execution of the asynchronous task is not necessarily guaranteed), would not be concurrent (there is no overlap in the execution periods), yet would be asynchronous.

Oh, by the way, in some platforms the system can decide to inline the asynchronous task, and thus execute it right there as a synchronous operation (assuming it can, that is not viable for every task).

Again, asynchronous just means you do not know when it will happen.

---

You may also be interested in The difference between “concurrent” and “parallel” execution?.

Answer 3

Asynchronous: not existing or occurring at the same time.

Synchronous: existing or occurring at the same time.

The async attribute means that the script will be executed as soon as it is downloaded even if the html is still parsing, which means both processes exist at the same time to me.

This is indeed confusing!

Consider instead the meanings of synchronized and unsynchronized. Two things are synchronized if the timing of one depends on the other, and unsynchronized if their timings are unrelated.

In your example asynchronous workflow, we have two things happening: script execution and html parsing. These two things are unsynchronized; the timing of the execution operation and the timing of the parsing operation do not depend on each other. If we made the workflow synchronous, then the operations would become synchronized. Say, the execution does not begin until the parsing has definitely ended.

But it is important to realize that there are actually three possibilities here:

• execution and parsing are truly unsynchronized; they can both happen at any time and in any order, whatever is most efficient.
• execution and parsing are synchronized; say, the CPU does nothing while waiting for the I/O to complete.
• execution and parsing are synchronized, but while we are waiting for the I/O to complete, the CPU is permitted to do other work, and then come back and do the parsing after the download completes.

Once you grasp that, the purpose of async workflows in popular programming languages becomes more clear:

• Asynchronous workflows help us implement high-latency operations efficiently because we are not constrained to order unrelated things in time. If parsing is high-latency and I/O bound, and script execution is high-latency and CPU bound, we can get a win by fully or partially unsynchronizing those operations.

• The await operator in languages like C# is the ordering operation on asynchronous workflows. An await is an asynchronous wait; it is a point where we express an ordering relationship between two parts of an asynchronous workflow, and say that there is a "happens before" relationship between the code before the await and the code after the await. This is how we implement the third option.

If that is all too abstract, think of some real-world examples. When you send a letter -- a high-latency I/O bound operation -- you can still do CPU-intensive work -- math homework, say -- while you are waiting to get a reply to your letter. The operations of do your math homework and read your mail are unsynchronized.

But suppose now you send a letter and the reply contains a number you need to do your taxes. Now you cannot do the CPU work -- computing your taxes -- until the I/O operation completes. But you can still mow the lawn while you are waiting. That's an asynchronous workflow that has expressed a timing relation between its parts.

Answer 4

I'm an electrical engineer, and we dealt with synchronous vs asynchronous in logic circuits (logic gates).

Let's say you have an AND-gate (or any gate), which has two inputs, and an output.

If it is asynchronous, it will update its output the moment any of the inputs change in such a way that the output changes. This is how your example worked - the program you mentioned.

However, if that gate also has a clock (for example, a 1 second period square wave) attached to it, where it updates on the beat of each second, as the square wave goes from low to high, it is synchronous. It is bound to the frequency of the clock. So it is synchronous. You could wire that clock to many circuits, and they would operate in rhythm with each other - synchronized. If your program only checked whether it was read to run every second, it would also be synchronized.

Answer 5

Concrete Examples

I would like to add some real-world examples, and connect them to the software engineering world. First, consider something that I hope matches your intuitive definition of "synchronous": the flashing of fireflies, under some circumstances. Second, consider the 4x100 women's Olympic relay race. Third, consider that old trope from military films: "Men, synchronize your watches!"

Now, let's think about what's going on. Let's start out by observing that all of these things are processes, or entities extended in time. It doesn't make sense to say that a bowl is "synchronous" and rock is "async." Second, it takes two to tango. You can't say that "a runner is sync". Sync with what? Finally, in order for two processes to do something at the same time, unless they already have the exact same frequency and phase, one or both of them must wait.

Analysis

When the dictionary definition says two entities in sync "occur or exist at the same time", that aligns very nicely with the concept of the light from fireflies. Unfortunately, saying that the light is "in sync" is a sloppy way of saying that the firefly lighting processes are synchronized.

So how can a bunch of fireflies, which presumably don't have Apple SmartWatch and NTP to guide them, manage to flash their rear ends at the same time? Well, it's pretty easy if they have a means to set a consistent tempo and can make small adjustments to it. They just flash, and if more folks flash right after them, they slow down (increase the delay), whereas if more flash right before them, they speed up (decrease the delay). So they can use a simple feedback process to arrive at essentially the same tempo and phase. The important observation here is to note that they achieve synchrony by waiting for the right moment to flash.

The 4x100 race is interesting because you see both forms of process timing in action: the runners within a team are synchronized, while the runners on different teams are "async". The second runner in the relay must wait until the first runner enters the transfer zone. The hand-off is a synchronous event between those two runners. However, the runners in different lanes don't care what's happening in another lane, and most certainly don't slow down and do their hand-offs in sync. Each lane of runners is asynchronous with respect to each other. Again, we see that synchronization entails waiting, while asynchrony does not.

Finally, the soldiers in a company (platoon, fire team, etc.) must synchronize their watches so that they can attack the enemy at the same time. It might be that some soldiers arrive at their positions before others, or have an opportunity to fire on the enemy sooner. But a simultaneous attack is generally more effective than a haphazard attack because of the element of surprise. So to achieve synchrony, many of the soldiers must wait for the appointed time to act.

Defining Feature

Why this emphasis on waiting? Well, it's because waiting is the defining feature which distinguishes synchronous from asynchronous processes. If you have two processes which you know nothing about, you should, by default, assume that they are asynchronous. For example, a package delivery and an ambulance driving by are most likely not synchronized. In order to demonstrate that two processes are, in fact, synchronized, you need to find a very special moment in time: the synchronization point.

A delivery driver dropping off a package and an ambulance rushing someone to the hospital don't generally share any points in time that we identify as a "synchronization point". On the other hand, fireflies flashing in unison have a sync point every time they flash, relay runners have a sync point every time they hand off the baton, and soldiers have a sync point when they launch their attack. If you can identify one or more sync points, then the processes are synchronized. This should be easy to understand, because "syn-" is a Greek prefix meaning "with" or "together", and "chrono" is the Greek root for "time". "Synchronized" literally means "at the same time", and you should think of it as identifying the existence of synchronization points.

Boundaries

Note that "synchronization" does not necessarily apply to the entire lifetime of either or both processes. I would argue that it only applies to "the waiting time up to and including the synchronization point(s)". Thus, two processes may operate asynchronously until they reach a state where they need to communicate, then they become synchronized, exchange information, and afterwards continue asynchronously. A simple example is meeting someone for coffee. Obviously, the meeting is a synchronization point (or many, rather), and the fact that two people arrive at that point demonstrates the synchrony. However, we would not say that because two people met for coffee, those two human lifetimes are "synchronized". It may be that was the only instant in their lives that they met, and everything else they do is otherwise independent.

It is also not the case that incidental meets demonstrate synchrony. If two strangers pass each other on the street, the fact that they are in a particular place at some time does not prove synchrony. Nor does the fact that one person is sitting on a bench waiting for the bus, and another happens to walk by. Processes are only synchronous when they meet for a purpose.

Software Connection

Now, let's think about a very fundamental task in software: reading from a file. As you probably know, mass storage is usually thousands to millions of times slower than cache or main memory. For this reason, operating systems and programming language libraries generally offer both synchronous and asynchronous I/O operations. Now, even if your program only has a single thread, you should think of the OS as being a "separate process" for the purposes of this discussion.

Sync

When you make a "synchronous I/O read", your thread must wait until the data is available, at which point it continues. This is very much like a relay runner handing the baton off to the next runner, but imagine instead a relay with only two runners going all the way round the track, and the second runner also hands off back to the first.

In this case, your program thread and the OS I/O process are not "happening (acting) at the same time", and so it seems weird to say that these processes are "synchronized". But that's the wrong way to look at it! That's like saying: "The runners on a relay team aren't running at the same time, so they aren't synchronized." In fact, both statements are wrong! The runners on a relay team do and must run at the same time, but only at a very specific moment: the hand-off of the baton. In fact, it is only this special moment during the race that convinces us that relay teams are synchronized to begin with! If we view the I/O request and response as "the baton", then it is easy to see that blocking I/O is essentially isomorphic to a 2-woman relay race.

On the other hand, if we think about something like Finite Element Analysis on a supercomputer, we see that thousands of processes must work in lock-step to update a massive global state. Even if some of the nodes complete their work for a given time-step before others, they all need to wait for the time step to complete because the results propagate to neighbors through space. This kind of synchronization is like the fireflies: all actors are performing the same kind of task.

Process Variety

For this reason, we can invent a few terms to help us see that there are three kinds of things going on: "homogeneous synchrony", "heterogeneous synchrony", and "sequential synchrony". So when the actors are performing the same task simultaneously (FEA, fireflies), they are "homogeneous". When they are performing different tasks simultaneously (soldiers running vs. crawling vs. swimming to their destinations, physics vs. sound vs. AI threads in a game), they are "heterogeneous". When they are performing tasks one at a time, they are "sequential" (relay runners, blocking I/O). They may look very different, but they share one essential property: all types of actors perform some waiting to ensure that everyone arrives at the synchronization point at the same time. Whether the actors are "acting" or waiting in between synchronization points, or "performing the same action" is irrelevant to the property of synchronicity.

The render pipelines in a GPU are synchronous because they all must finish the frame together, and start a new frame together. They are homogeneous because they are doing the same kinds of work, and they are all active together. But the main game loop of a server and the blocking I/O threads which process remote input are heterogeneous because they do very different kinds of work, and some of the I/O threads won't be doing anything at all, because not all the connections are used. Even so, they are synchronized, because they must share state atomically (a player must not see a partial game world update, nor must the server see only a fragment of player input).

Async

Now, let's consider an "async I/O read". When your program sends a request to the OS to read a bit of data from storage, the call returns immediately. Let's ignore callbacks and focus on polling. In general, the moment that data is available to your program does not correspond to any special point in time as far as your program's thread is concerned. If your program isn't explicitly waiting for the data, then the thread won't even know exactly when that moment occurs. It will only discover that data is waiting the next time it checks.

There is no special meeting time where the OS and the program thread agree to hand over the data. They are like two ships passing in the night. Asynchrony is characterized by this absence of waiting. Of course, the program thread will often end up waiting on the I/O operation after all, but it doesn't need to. It can happily go on doing other calculations while the I/O fetch is occurring, and only check later when it has a moment to spare. Of course, once the OS is done fetching data, it doesn't sit around waiting, either. It just puts the data somewhere convenient and goes on about its business. In this case, it's like the program hands the baton off to the OS, and the OS comes around later, drops the baton on the ground along with the data, and walks off the track. The program may or may not be waiting around to receive the hand-off.

Parallelism

When we mark a function as "async" in software, it often means we want parallelism. But remember that parallelism does not imply synchrony. The fireflies are a good example, because they, too exhibited both synchronous and asynchronous behavior. While most of the flies flashed in unison, many were obviously out of tune with the rest of the group and flashed more randomly. The flies may have been acting simultaneously, but they were not all synchronized.

Now when we mark some code as "async", it looks funny, because it implies that the rest of the code not so marked is "sync". What does that even mean? Didn't we insist that "synchronization" required two to tango? But what if we are talking about code executing in a single thread? In this case, we need to take a step back and think about a program as a sequence of states and transitions between those states. A statement in a program causes a state transition. We can think of it as a "micro-process" that starts and stops with the statement. The sequence points defined by the language are, in fact, the synchronization points of these "micro-processes". And thus, we can view a single-threaded, serial program as an example of sequential synchrony.

The integrity of the programming language guarantees that state updates don't interfere across statements, and the sequence points define boundaries across which the compiler is not allowed to make observable optimizations. For instance, the order of evaluation of expressions within a statement might be undefined or underspecified, giving the compiler freedom to optimize the statement in a variety of ways. But by the time the next statement begins, the program should be in a well-defined state, if the PL itself is sound.

By now, it should be clear what we mean by "async". It means exactly that the implied contract of synchrony within a block of code is exempted for the async block. It is allowed to update the program state independently, without the guarantees of safety normally implied by the sequential(ly consistent, synchronous) computation model. Of course, this means that we need to take special care that we don't destroy the program state with inconsistency. This usually means that we introduce limited, explicit synchrony to coordinate with the async block. Note that this means the async block can be both asynchronous and synchronous at different times! But recalling that synchronization merely indicates the existence of a sync point, we should have no trouble accepting this notion.

Answer 6

Imagine two satellites orbiting Earth.

• Satellite A has a period of rotation around Earth such that for every full rotation of the planet, the satellite has gone around Earth more or less than one time.
• Satellite B has a period of rotation around Earth such that for every full rotation of the planet, the satellite has gone around Earth exactly one time.

Satellite B in the example above is in geosynchronous orbit as defined by

having a period of rotation synchronous with that of the earth's rotation.

One does not argue that satellite A is geosynchronous simply because it "exists or occur[s] at the same time" as the planet. In fact, satellite B itself is not what is relevant either - what is relevant is the period of rotation that is synchronized to the period of rotation to that of Earth. It's not about the simultaneous existence of the objects; it's about the relationship between the objects. Hold this thought.

Suppose I tell you two threads on a system are running at the same time. Thread A (TA) is fetching data for Process A and Thread B (TB) is fetching data for Process B. I ask you, "Are TA and TB asynchronous?". Your response would be, "How could I know? Id have to see the code that invoked them in their respective processes." To which I would retort in my attempt to be tricky, "But I'm telling you that TA and TB are definitely running at the same time."

And you, being quite the clever individual, would respond, "Again - they may be running concurrently but I have no clue if they are running asynchronously with regards to their respective processes that invoked them. TA and TB running asynchronously to each other really makes no sense because they weren't spawned from the same process."

So by now we should be gaining some intuition that the existence of a relationship is what is relevant here, not just the existence of these two threads themselves. When a method is executed asynchronously, what we're saying is that the execution of that method does "NOT need to exist or occur at the same time" as the execution of the method that invoked it. Take the following example:

func Invoker() {
    DoThis();
    DoThatAsync();

    var foo = CheckThis();
    ... do some work ...
    CheckThat(foo);

    await DoThatAsync();

    CheckThat();
}

From our discussion of satellites earlier, "It's not about the simultaneous existence of the objects; it's about the relationship between the objects." It's not about the existence of an invoking method and the invoked method; it's about the existence of a relationship between the execution of invoker and the execution of the invoked. If we looked at our system threads and found that DoThatAsync() was invoked but wasn't executing, maybe it's waiting on the scheduler or some other I/O, that doesn't necessarily mean that the invoking method Invoker() isn't executing - there's work it could be doing. Sure, it may be at the point of awaiting DoThatAsync() , but that is not guaranteed. This is not true of the other functions once they've been invoked - if they halt, Invoker() halts - no matter what. This is guaranteed. The execution between the Invoker() and invoked synchronous method "exists or occurs at the same time".

Answer 7

One way to think about it are SIMD instructions, like AVX. Here are some examples of how they are used.

Synchroneous SIMD instructions allow you to do multiple calculations at precisely the same time, in the same thread, by operating a Single Instruction on Multiple Data.

While asynchroneous multithreading allows you to do multiple calculations at "probably" "somewhat" "similar" times.

Combine this with the following definitions:

synchronous adjective syn·​chro·​nous | \ ˈsiŋ-krə-nəs , ˈsin-

1 : happening, existing, or arising at precisely the same time [emphasis mine]

asynchronous adjective asyn·​chro·​nous | \ (ˌ)ā-ˈsiŋ-krə-nəs , -ˈsin-\

1 : [...] : not synchronous

Answer 8

"synchronous" means that two events occur at the same time - but which events?

When we say "synchronous execution", we mean that the caller and callee are executing (i.e. on stack) at the same time. That's probably the meaning you are after.

When we say "synchronous logic gate", we mean that the logic gate is synchronized with the cpu clock.

When we say "synchronous model" in the context of distributed systems, we mean that all nodes execute their programs in lock-step, and messages sent in step n are guaranteed to arrive at the start of step n+1.

When the Java Language Specification says that a thread "synchronizes with" another, it means that the actions in the different threads occur "at the same time" (with respect to the happens before relationship). And when they say that two threads "synchronize access to an object", they actually mean that the threads synchronize with each other to ensure they never work on the object at the same time.

... and I'm pretty sure you could apply the word in even more contexts, because "things happen at the same time" is quite generic an idea :-)

Answer 9

An analogy that made me understand the difference between Sync vs Async vs Multi-threaded is that of a cook in the kitchen.

Imagine you are making pasta. You have three steps:

1. Boil and drain the pasta
2. Prepare sauce
3. Combine pasta and sauce

Synchronous method. In synchronous scenario there is only one person (thread) doing all the work in sequence. First you boil the pasta and you stand there watching it boiling. Then you drain it and set it aside. Then you prepare the sauce. When sauce is ready you take the pasta, mix it with the sauce and your dish is ready. The problem here is it's inefficient. Because you were working sequentially in a synchronous manner, you couldn't work on the sauce while the pasta was boiling. So, it made you waste time, and your pasta got cold while the sauce was being prepared.

Asynchronous method. In this scenario, there is still only one cook (thread), but while the pasta is boiling you go and make your sauce. When pasta is boiled, you are called-back from making sauce to drain it, and then you are called-back again to finish the sauce. This is more efficient now, because you saved time and your pasta didn't have to wait for the sauce that long.

Multi-threaded method. Now, imagine you hire a new cook. Now you have two cooks (threads). While one cook is doing pasta, second cook is making sauce. Is it necessary in this scenario? No, because making pasta is simple enough to be efficient with asynchronous method. And, managing multiple cooks is additional overhead. But if you were making more complicated dish or more dishes at once, multiple cooks are useful.

Answer 10

A good question, and terms that are often used in different ways that lead to confusion.

My answer is that these terms are relative — and what they are relative to is the main program that is executing (or sometimes to a thread).

These terms specify something about the internal operation & timing of a program, as to whether messages are sent or received in a blocking way (sync) or in a non-blocking way (async).  If a (the main) thread is blocked by sending or receiving, that is "sync" and if it is interruptuble somehow then it is "async".  To reiterate, these terms are about implementations that both do (regular) work as well as handle events.

(IMHO, of course) once a message is on the wire, there is no such thing as sync vs. async.  In messaging there is a sender and a receiver, each of them can have a sync or async implementation independent of the other — but once a message is on the wire, it is just a message, no longer sync or async.  We may classify a message as a request or reply or one way message, but that is orthogonal to sync and async (which refer to whether the implementation is blocking waiting or can be interrupted in some way).

Answer 11

I think the key of your confusion can be summed up by:

The async attribute means that the script will be executed as soon as it is downloaded even if the html is still parsing

The thing to realise is that this sentence does not make sense because it describes an impossible situation. If the HTML is still parsing then the script download process would not even begin if it's asynchronous.

In programming, synchronous means:

All the data you are interested in already exist in memory at the time you are executing your logic

While asynchronous means:

Some of the data you are interested in does not exist yet and only exist at some point in the future

Indeed, it is this not-the-present aspect of asynchronous programming that usually confuses people.

How scripts are normally loaded is the html parsing is paused, then the script is downloaded, when the script download completes it is executed and then the html parsing continues. The html parsing and script execution happens at the "same" time (same time meaning together, not simultaneous).

How async scripts are loaded is the html sees the script tag and then remembers to download the script in the future but continues to parse. The html parsing is not paused for script downloading. Later, after the html parsing completes all the async scripts are downloaded and executed. The html parsing and script execution does not happen at the same time (again, same time meaning together, in this case they are executed separately).

So to summarise:

• Synchronous scripts are parsed together with the html.

• Asynchronous scripts are parsed separately in the future.

So the definition of the async property is not that the script is executed as soon as it is downloaded - this is true for both synchronous and asynchronous scripts. The definition of async is the html parsing does not wait for the script to download.