Load testing : how to generate per second requests?

Question

I have a server component which runs over Zeroc-ICE. When I wanted to load test it, I thought that using parallel library to create multiple requests would do it. But it dint end up that way. Using Parallel(Parallel.For) library from C# apparently was easier but it doesn't seem to be exactly generating everything parallel in the same instant. So it cannot be the definition for creating N requests per second. How should I do it ? I guess anyone who wants to do load testing first would actually think about this .

1. What's the efficient way to actually create N requests in really per second ?

2. Another myth is about the parallel programming. Please enlighten us , if you have used parallel programming patterns in C# or .Net in general . Imagine I have 5 processes. How will start all the five processes at the same time. What does it mean to my consumption of resources ? I have tried reading into many of the material available over the net but I get more and more questions than them being the answer to my questions.

3. I used Parallel.For and created N threads and measured time. Then I tried the same thing using Task.Factory.start for enumeration of tasks. The time measured was different. So What exactly is the different between using these ? When I should use the corresponding classes and for what purposes exactly ? we often have lots of riches but its just we exactly don't know how to differentiate one from the another. This is one such case for me, not being able to find why I should not use one from the another.

4. I used the stopwatch class to measure these times which claims to be the best. In the scenario of me load testing a component, what would be the way to measure the time of response. Stopwatch seems to be the best solution for me . Any opinions are welcome.

ps: There are many load testing tools for web applications. Mine is a customised case of server components. And my question is more pertaining to creating N threads per second.

All opinions are welcome. Just don't think its so much not a programming question. It ofcourse is. It should ring bells for any programmer who wants to QE stuff by himself to know the performance of his product , first hand by himself.I have tried many options and then had to fall back on how should I actually do it ?

Answer 1

I don't have all the answers. Hopefully I can shed some light on it.

To simplify my previous statements about .NET's threading models, just know that Parallel Library uses Tasks, and the default TaskScheduler for Tasks, uses the ThreadPool. The higher you go in the hierarchy (ThreadPool is at the bottom), the more overhead you have when creating the items. That extra overhead certainly doesn't mean it's slower, but it's good to know that it's there. Ultimately the performance of your algorithm in a multi-threaded environment comes down to its design. What performs well sequentially may not perform as well in parallel. There are too many factors involved to give you hard and fast rules, they change depending on what you're trying to do. Since you're dealing with network requests, I'll try and give a small example.

Let me state that I am no expert with sockets, and I know next to nothing about Zeroc-Ice. I do know about bit about asynchronous operations, and this is where it will really help you. If you send a synchronous request via a socket, when you call Socket.Receive(), your thread will block until a request is received. This isn't good. Your thread can't make any more requests since it's blocked. Using Socket.Beginxxxxxx(), the I/O request will be made and put in the IRP queue for the socket, and your thread will keep going. This means, that your thread could actually make thousands of requests in a loop without any blocking at all!

If I'm understanding you correctly, you are using calls via Zeroc-Ice in your testing code, not actually trying to reach an http endpoint. If that's the case, I can admit that I don't know how Zeroc-Ice works. I would, however, suggest following the advice listed here, particularly the part: Consider Asynchronous Method Invocation (AMI). The page shows this:

By using AMI, the client regains the thread of control as soon as the invocation has been sent (or, if it cannot be sent immediately, has been queued), allowing the client to use that thread to perform other useful work in the mean time.

Which seems to be the equivalent of what I described above using .NET sockets. There may be other ways to improve the performance when trying to do a lot of sends, but I would start here or with any other suggestion listed on that page. You've been very vague about the design of your application, so I can be more specific than I have been above. Just remember, do not use more threads than absolutely necessary to get what you need done, otherwise you'll likely find your application running far slower than you want.

Some examples in pseudocode (tried to make it as close to ice as possible without me actually having to learn it):

var iterations = 100000;
for (int i = 0; i < iterations; i++)
{
    // The thread blocks here waiting for the response.
    // That slows down your loop and you're just wasting
    // CPU cycles that could instead be sending/receiving more objects
    MyObjectPrx obj = iceComm.stringToProxy("whateverissupposedtogohere");
    obj.DoStuff();
}

A better way:

public interface MyObjectPrx : Ice.ObjectPrx
{
    Ice.AsyncResult GetObject(int obj, Ice.AsyncCallback cb, object cookie);
    // other functions
}

public static void Finished(Ice.AsyncResult result)
{
    MyObjectPrx obj = (MyObjectPrx)result.GetProxy();
    obj.DoStuff();
}

static void Main(string[] args)
{
    // threaded code...
    var iterations = 100000;
    for (int i = 0; i < iterations; i++)
    {
        int num = //whatever
        MyObjectPrx prx = //whatever
        Ice.AsyncCallback cb = new Ice.AsyncCallback(Finished);
        // This function immediately gets called, and the loop continues
        // it doesn't wait for a response, it just continually sends out socket
        // requests as fast as your CPU can handle them.  The response from the
        // server will be handled in the callback function when the request
        // completes.  Hopefully you can see how this is much faster when 
        // sending sockets.  If your server does not use an Async model 
        // like this, however, it's quite possible that your server won't 
        // be able to handle the requests
        prx.GetObject(num, cb, null);
    }
}

Keep in mind that more threads != better performance when trying to send sockets (or really doing anything). Threads are not magic in that they will automatically solve whatever problem you're working on. Ideally, you want 1 thread per core, unless a thread is spending much of its time waiting, then you can justify having more. Running each request in its own thread is a bad idea, since context switches will occur and resource waste. (If you want to see everything I wrote about that, click edit and look at the past revisions of this post. I removed it since it only seemed to cloud the main issue at hand.)

You can definitely make these request in threads, if you want to make a large number of requests per second. However, don't go overboard with the thread creation. Find a balance and stick with it. You'll get better performance if you use an asynchronous model vs a synchronous one.

I hope that helps.

Answer 2

I'm going to skip over question 1) and get right into #2, since that's generally an acceptable way to accomplish what you're looking for. In the past to achieve n messages per second you can create a single process that will then launch p AppDomains. Each AppDomain basically just starts running a request loop once a certain point in time has been reached (using a Timer). This time should be same for each AppDomain to ensure that they start hitting your server at the same point in time.

Something like this should work for sending your requests:

WaitCallback del = state => 
{ 
    ManualResetEvent[] resetEvents = new ManualResetEvent[10000]; 
    WebClient[] clients = new WebClient[10000]; 

    for (int index = 0; index < 10000; index++) 
    { 
        resetEvents[index] = new ManualResetEvent(false); 
        clients[index] = new WebClient(); 

        clients[index].OpenReadCompleted += new OpenReadCompletedEventHandler (client_OpenReadCompleted); 

        clients[index].OpenReadAsync(new Uri(@"<REQUESTURL>"), resetEvents[index]); 
    } 

    bool succeeded = ManualResetEvent.WaitAll(resetEvents, 10000); 
    Complete(succeeded); 

    for (int index = 0; index < 10000; index++) 
    { 
        resetEvents[index].Dispose(); 
        clients[index].Dispose(); 
    } 
}; 

while(running)
{
    ThreadPool.QueueUserWorkItem(del);
    Thread.Sleep(1000);
}

This will probably crush performance on whichever machine you're running it on, so you could always implement a simmilar type of loop from several different machines if you have the resources (using processes instead of app domains).

For your third question, give this link a read http://www.albahari.com/threading/

Finally, a stopwatch should be paired with a hit counter to track both duration & unique hits on your server. That should let you perform some analysis after the fact.

Answer 3

Don't bother with threads, if N is reasonably small. To generate N requests per second, use the wall clock time (DateTime.Now). Take the time both before and after the request, then add a Sleep to delay the next request.

For instance, with N=5 (200 ms):

Before request: 12:33:05.014
After request: 12:33:05.077
Sleep(137)
Before request: 12:33:05.214
After request: 12:33:05.271
Sleep(131)

This is not perfect; you may find that Sleep isn't exact. You can keep a running count of deviations (before the X'th requests, the time should be X-1/N later) and adjust the Sleep period accordingly.

Once N becomes too large, you simply create M threads and let each thread generate N/M requests in the same fashion.

Answer 4

The easiest way to achieve load testing for any .NET project is to purchase the Ultimate edition of Visual Studio. This comes with an integrated testing tools to help preform all kinds of tests including load tests. Load tests can be preform by creating virtual users either on a single PC or distributed across several for larger numbers of users, there is also a small program that can be installed on the target servers to return additional data for the duration of the test.

This is expensive though, but the ultimate edition comes with a lot of features, so if all were used it would be a more reasonable price.

Answer 5

If you purely want to have X threads all hit your resource at the exact same time you can put each thread behind a countdown latch and specify a short wait period between the semaphore checks.

C# has an implementation (http://msdn.microsoft.com/en-us/library/system.threading.countdownevent(VS.100).aspx).

At the same time, if you are stress testing your system you might actually want to check for race conditions as well, in which case you would want to setup thread sleep periods on each thread that osculate over time with randomized frequency and peaks/furloughs.

Similarly you may not actually want to just rapidly send multiple requests, you might have better success at putting your server into a bad state/testing its real world performance by setting up a smaller number of threads that spend more time in consuming and sending messages back and forth over the socket, as your server likely is going to need to spin up its own threads to handle slow ongoing messages.