0

A while ago I read an article stating that overhead of an async/await call was around 50ms. More recently I read an article that it was around 5ms. I was having a discussion about whether we should standardize async operations for all DB access and decided to take a crack at measuring it myself. And I ended up adding the following methods to a controller:

private int profileIterations = 1000;
[HttpGet]
public long NonAsyncLoop()
{
    var timer = new System.Diagnostics.Stopwatch();
    timer.Start();
    for (int i = 0; i < profileIterations; i++)
    {
        Thread.Sleep(5);
    }
    timer.Stop();
    return timer.ElapsedMilliseconds;
}

[HttpGet]
public async Task<long> AsyncLoop()
{
    var timer = new System.Diagnostics.Stopwatch();
    timer.Start();
    for (int i = 0; i < profileIterations; i++)
    {
        await Task.Delay(5);
    }
    timer.Stop();
    return timer.ElapsedMilliseconds;
}

This test returns surprisingly regular results indicating that the overhead of calling await Task.Delay() vs Thread.Sleep() is ~1/3 of a ms. Does anyone have any other easy test that could indicate the overhead? Because below 10ms of overhead it becomes a no-brainer to standardize async operations for all DB access.

The other devs and I are deciding about whether to just use async for all DB operations going forward, the app we're working with has all synchronous DB IO, so there is a possibility we may be converting large amounts of existing access when the opportunity arises.

To get an idea of DB latency I'm looking at app insights request profiles and looking at items listed under "SQL (Azure Database) Activities". Most of these "activities" show wait times of between 1 and 5ms. So if my test is roughly accurate, then using async/await for all DB access is clearly the way to go moving forward, and existing synchronous access should be converted whenever practical. However, if overhead is in the 5-50ms range, then using async would actually decrease thread availability while making response time slower simultaneously.

24
  • 1
    What routine database operation over a network do you know of that takes less than 100ms? At 50ms for an async/await call, it's a wash, because the underlying thread is freed up to do other work during the remaining 50ms. Mar 4, 2021 at 21:09
  • 7
    Where you might be remembering 50ms from is that's what Microsoft states as their recommended threshold for making a method asynchronous. See blog.stephencleary.com/2013/04/ui-guidelines-for-async.html Mar 4, 2021 at 21:16
  • See also stackoverflow.com/q/38118051 Mar 4, 2021 at 21:18
  • Do you have a reference for this? It's not clear to me what the source of this overhead would be aside from the event loop. Perhaps you misunderstood? As Robert Harvey points out with thresholds, the point of asyncio is to avoid blocking on IO. But if your IO is super-fast, it could be finished long before the loop comes back around for the response. That might be OK if you are waiting on multiple responses, though.
    – JimmyJames
    Mar 4, 2021 at 21:44
  • 4
    I applaud your curiosity, I’m disappointed that others are so quick to dismiss a valid question like this. But to your latter statement: It’s not faster, that’s certain. It’s actually doing strictly more work than doing the IO in the simple blocking way. It just happens to free up in the thread while it waits, that’s the advantage. So technically speaking, you can get infinitesimally lower latency with a blocking call, but at a much higher resource cost (a blocked thread)
    – Alexander
    Mar 5, 2021 at 3:10

1 Answer 1

4

Overhead is less than 0.1 ms

The rest of the answer is how I got that.

I decided to introduce a version of the code that uses Task.Run(()=> Thread.Sleep(waitTime)) which I consider a fairer comparison to simply sleep. So I have three versions.

I'm also compute how much time the code is not waiting. That is, I define how many iterations profileIterations, how much to wait waitTime, and then I multiply them so I know how much time is supposed to be expended waiting… Subtract that from the total time. Then I profileIterations so I know how much time is not waiting per iteration.


This is the code:

Sleep version

int profileIterations = 1000;
int waitTime = 5;
var timer = new System.Diagnostics.Stopwatch();
timer.Start();
for (int i = 0; i < profileIterations; i++)
{
    Thread.Sleep(waitTime);
}
timer.Stop();
Console.WriteLine((timer.ElapsedMilliseconds - waitTime * profileIterations) / (double)profileIterations);

Task.Run version

int profileIterations = 1000;
int waitTime = 5;
var timer = new System.Diagnostics.Stopwatch();
timer.Start();
for (int i = 0; i < profileIterations; i++)
{
    await System.Threading.Tasks.Task.Run(()=> Thread.Sleep(waitTime));
}
timer.Stop();
Console.WriteLine((timer.ElapsedMilliseconds - waitTime * profileIterations) / (double)profileIterations);

Delay version

int profileIterations = 1000;
int waitTime = 5;
var timer = new System.Diagnostics.Stopwatch();
timer.Start();
for (int i = 0; i < profileIterations; i++)
{
    await System.Threading.Tasks.Task.Delay(waitTime);
}
timer.Stop();
Console.WriteLine((timer.ElapsedMilliseconds - waitTime * profileIterations) / (double)profileIterations);

And these are the times:

  • Sleep version: 10.605
  • Task.Run version: 10.584
  • Delay version: 10.814

Task.Run(()=> Thread.Sleep(waitTime)) appears to be faster than Thread.Sleep(waitTime), which is nonsensical. This measurement is clearly wrong.

Yeah, it is waiting around 15 to 16 milliseconds instead of 5 (it is waiting around 10 extra milliseconds). Because time resolution is a thing. See Can I improve the resolution of Thread.Sleep? So I tried with waitTime = 16, got these:

  • Sleep version: 14.181
  • Task.Run version: 14.82
  • Delay version: 15.113

Now it is waiting about 30 milliseconds? Let us set waitTime = 30, got these:

  • Sleep version: 1.288
  • Task.Run version: 1.432
  • Delay version: 1.561

Ha! - Let us do better. Change timer.ElapsedMilliseconds (long) to timer.Elapsed.TotalMilliseconds (double), and we have better resolution.

Got these:

  • Sleep version: 1.3736827000000011
  • Task.Run version: 1.4033598999999994
  • Delay version: 1.7195541999999986

Not all of those digits are significant. Stopwatch can't report below 0.0001 milliseconds (100 nanoseconds) in my computer.

It appears that Delay is slower always. It is not a fair comparison anyway. Between the other two… This measurements suggest an overhead of 0.03 to 0.1 milliseconds from Task.Run and await.


Let us replace Thread.Sleep with Thread.SleepWait, this time we will have the thread spin!

This is the new code:

SpinWait version

int profileIterations = 1000;
int iterations = 1000;
var timer = new System.Diagnostics.Stopwatch();
timer.Start();
for (int i = 0; i < profileIterations; i++)
{
    Thread.SpinWait(iterations);
}
timer.Stop();
Console.WriteLine(timer.Elapsed.TotalMilliseconds / (double)profileIterations);

New Task.Run version

int profileIterations = 1000;
int iterations = 1000;
var timer = new System.Diagnostics.Stopwatch();
timer.Start();
for (int i = 0; i < profileIterations; i++)
{
    await System.Threading.Tasks.Task.Run(()=> Thread.SpinWait(iterations));
}
timer.Stop();
Console.WriteLine(timer.Elapsed.TotalMilliseconds/ (double)profileIterations);

And these are the new results:

  • SpinWait version: 0.0410909
  • New Task.Run version: 0.053339

Increased to iterations = 10000, got these:

  • SpinWait version: 0.3893809
  • New Task.Run version: 0.4080467

That settles it for me. The overhead is between 0.01 and 0.02 milliseconds. Including both Task.Run and await.

I tried increasing the iterations even more, and that gave me a larger overhead. I believe at that point I'm no longer measuring overhead but the interference of other processes. However, if you are curious, it only got up to 0.06 milliseconds. Which is still below the 0.1 milliseconds I got using Sleep.


Finally, I want to mention that an async method that finished synchronously will have less overhead that one that did not. Similarly using ValueTask instead of Task can reduce overhead and allocations.


This tests were done in Windows 10, .NET 5.0, Intel I3. Oh, and using LINQPad. Yeah, LINQPad.

3
  • Yeah, tried same kinda thing with File IO. Results were pretty similar. Cost can only be tenths of ms at most.
    – Ian
    Mar 5, 2021 at 21:22
  • @Ian Ok, that is another thing. IO task (file, network, etc...) are promise tasks. They setup an event that the OS calls when the operation is done. And the event handler will start the continuations of the task. Which is similar to Delay, actually, except there the event is a timer. As seen here Delay takes longer that Task.Run(Sleep), and that is not from async, await, or Task. It seems to be the event. I'd argue that this is the right way to do IO, because most of it will be some device using a buffer via DMA and not the CPU... Meanwhile the thread is free to do other stuff.
    – Theraot
    Mar 5, 2021 at 22:31
  • I agree. I think the old 50ms rule-of-thumb is completely invalid. Even if your DB reads are a couple ms, you still gain thread availability.
    – Ian
    Mar 6, 2021 at 0:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.