14

I have a server component which runs over Zeroc-ICE. When I wanted to load test it, I thought that using parallel library to create multiple requests would do it. But it dint end up that way. Using Parallel(Parallel.For) library from C# apparently was easier but it doesn't seem to be exactly generating everything parallel in the same instant. So it cannot be the definition for creating N requests per second. How should I do it ? I guess anyone who wants to do load testing first would actually think about this .

  1. What's the efficient way to actually create N requests in really per second ?

  2. Another myth is about the parallel programming. Please enlighten us , if you have used parallel programming patterns in C# or .Net in general . Imagine I have 5 processes. How will start all the five processes at the same time. What does it mean to my consumption of resources ? I have tried reading into many of the material available over the net but I get more and more questions than them being the answer to my questions.

  3. I used Parallel.For and created N threads and measured time. Then I tried the same thing using Task.Factory.start for enumeration of tasks. The time measured was different. So What exactly is the different between using these ? When I should use the corresponding classes and for what purposes exactly ? we often have lots of riches but its just we exactly don't know how to differentiate one from the another. This is one such case for me, not being able to find why I should not use one from the another.

  4. I used the stopwatch class to measure these times which claims to be the best. In the scenario of me load testing a component, what would be the way to measure the time of response. Stopwatch seems to be the best solution for me . Any opinions are welcome.

ps: There are many load testing tools for web applications. Mine is a customised case of server components. And my question is more pertaining to creating N threads per second.

All opinions are welcome. Just don't think its so much not a programming question. It ofcourse is. It should ring bells for any programmer who wants to QE stuff by himself to know the performance of his product , first hand by himself.I have tried many options and then had to fall back on how should I actually do it ?

13
  • The faq says if it pertains to a specific programming problem and if its a practical answerable problem in programming profession, it can be asked. people who are skeptic and flagging this. please comment.
    – King
    Jan 19, 2012 at 21:42
  • What do you mean by "the same instant"? I wonder if you can force TPL or PLinq in any way to achieve that. Jan 19, 2012 at 21:45
  • My question is about generating N requests per second. So same instant in this scenario was meant towards my understanding of using the parallel would start threads in a parallel fashion.
    – King
    Jan 19, 2012 at 21:48
  • Have you done any sequential analysis?
    – Lloyd
    Jan 19, 2012 at 21:49
  • 3
    It may pertain to programming, but there are just too many questions in your post (at least 4). I would reduce it to the one question you want to ask before it gets closed because it is too broad. Give relevant info, like the 10000 you only just mentioned, number of cores in your test machine). Showing code usually helps. Jan 19, 2012 at 22:04

5 Answers 5

10
+50

I don't have all the answers. Hopefully I can shed some light on it.

To simplify my previous statements about .NET's threading models, just know that Parallel Library uses Tasks, and the default TaskScheduler for Tasks, uses the ThreadPool. The higher you go in the hierarchy (ThreadPool is at the bottom), the more overhead you have when creating the items. That extra overhead certainly doesn't mean it's slower, but it's good to know that it's there. Ultimately the performance of your algorithm in a multi-threaded environment comes down to its design. What performs well sequentially may not perform as well in parallel. There are too many factors involved to give you hard and fast rules, they change depending on what you're trying to do. Since you're dealing with network requests, I'll try and give a small example.

Let me state that I am no expert with sockets, and I know next to nothing about Zeroc-Ice. I do know about bit about asynchronous operations, and this is where it will really help you. If you send a synchronous request via a socket, when you call Socket.Receive(), your thread will block until a request is received. This isn't good. Your thread can't make any more requests since it's blocked. Using Socket.Beginxxxxxx(), the I/O request will be made and put in the IRP queue for the socket, and your thread will keep going. This means, that your thread could actually make thousands of requests in a loop without any blocking at all!

If I'm understanding you correctly, you are using calls via Zeroc-Ice in your testing code, not actually trying to reach an http endpoint. If that's the case, I can admit that I don't know how Zeroc-Ice works. I would, however, suggest following the advice listed here, particularly the part: Consider Asynchronous Method Invocation (AMI). The page shows this:

By using AMI, the client regains the thread of control as soon as the invocation has been sent (or, if it cannot be sent immediately, has been queued), allowing the client to use that thread to perform other useful work in the mean time.

Which seems to be the equivalent of what I described above using .NET sockets. There may be other ways to improve the performance when trying to do a lot of sends, but I would start here or with any other suggestion listed on that page. You've been very vague about the design of your application, so I can be more specific than I have been above. Just remember, do not use more threads than absolutely necessary to get what you need done, otherwise you'll likely find your application running far slower than you want.

Some examples in pseudocode (tried to make it as close to ice as possible without me actually having to learn it):

var iterations = 100000;
for (int i = 0; i < iterations; i++)
{
    // The thread blocks here waiting for the response.
    // That slows down your loop and you're just wasting
    // CPU cycles that could instead be sending/receiving more objects
    MyObjectPrx obj = iceComm.stringToProxy("whateverissupposedtogohere");
    obj.DoStuff();
}

A better way:

public interface MyObjectPrx : Ice.ObjectPrx
{
    Ice.AsyncResult GetObject(int obj, Ice.AsyncCallback cb, object cookie);
    // other functions
}

public static void Finished(Ice.AsyncResult result)
{
    MyObjectPrx obj = (MyObjectPrx)result.GetProxy();
    obj.DoStuff();
}

static void Main(string[] args)
{
    // threaded code...
    var iterations = 100000;
    for (int i = 0; i < iterations; i++)
    {
        int num = //whatever
        MyObjectPrx prx = //whatever
        Ice.AsyncCallback cb = new Ice.AsyncCallback(Finished);
        // This function immediately gets called, and the loop continues
        // it doesn't wait for a response, it just continually sends out socket
        // requests as fast as your CPU can handle them.  The response from the
        // server will be handled in the callback function when the request
        // completes.  Hopefully you can see how this is much faster when 
        // sending sockets.  If your server does not use an Async model 
        // like this, however, it's quite possible that your server won't 
        // be able to handle the requests
        prx.GetObject(num, cb, null);
    }
}

Keep in mind that more threads != better performance when trying to send sockets (or really doing anything). Threads are not magic in that they will automatically solve whatever problem you're working on. Ideally, you want 1 thread per core, unless a thread is spending much of its time waiting, then you can justify having more. Running each request in its own thread is a bad idea, since context switches will occur and resource waste. (If you want to see everything I wrote about that, click edit and look at the past revisions of this post. I removed it since it only seemed to cloud the main issue at hand.)

You can definitely make these request in threads, if you want to make a large number of requests per second. However, don't go overboard with the thread creation. Find a balance and stick with it. You'll get better performance if you use an asynchronous model vs a synchronous one.

I hope that helps.

7
  • Why are you talking about performance so much? That doesn't seem to be what the OP wants.
    – svick
    Jan 20, 2012 at 7:57
  • 1
    @svick well the ops original post had 4 questions originally, and they asked questions about the performance of parallel vs tasks, then it was edited out, and now theyre back in. So, much of what you read was a result of that. Ultimately, though his question does have to do with performance, as he has the general idea correct, but is apparently lacking in his implementation. I believe my pointed answers at the end answer the question he didn't edit out. Jan 20, 2012 at 8:42
  • 1
    I was forced to reduce my questions because they wanted to vote it to close. Now it seems, its valid here to have them. @ChristopherCurrens +1 good point for difference with threadpool to tasks. That widened my understanding. But I'm still stuck how generating some N requests per second is really possible ? What exactly is the best way to do that ?
    – King
    Jan 20, 2012 at 9:29
  • @King - I guess I wasn't as clear as I thought I was. The last 3-4 paragraphs I thought would help you. I had assumed you were already using a loop of sorts already. If you were doing that, the problem is that your socket sends/receives are blocking, and thus slowing your requests down. Maybe I'll find some time to post some example pseudo code. Jan 20, 2012 at 17:13
  • I have no problem in actually sending them over ICE. The problem is what defines the implementation that would actually create N requests and something that can be said true to that number,N.
    – King
    Jan 20, 2012 at 17:59
2

I'm going to skip over question 1) and get right into #2, since that's generally an acceptable way to accomplish what you're looking for. In the past to achieve n messages per second you can create a single process that will then launch p AppDomains. Each AppDomain basically just starts running a request loop once a certain point in time has been reached (using a Timer). This time should be same for each AppDomain to ensure that they start hitting your server at the same point in time.

Something like this should work for sending your requests:

WaitCallback del = state => 
{ 
    ManualResetEvent[] resetEvents = new ManualResetEvent[10000]; 
    WebClient[] clients = new WebClient[10000]; 

    for (int index = 0; index < 10000; index++) 
    { 
        resetEvents[index] = new ManualResetEvent(false); 
        clients[index] = new WebClient(); 

        clients[index].OpenReadCompleted += new OpenReadCompletedEventHandler (client_OpenReadCompleted); 

        clients[index].OpenReadAsync(new Uri(@"<REQUESTURL>"), resetEvents[index]); 
    } 

    bool succeeded = ManualResetEvent.WaitAll(resetEvents, 10000); 
    Complete(succeeded); 

    for (int index = 0; index < 10000; index++) 
    { 
        resetEvents[index].Dispose(); 
        clients[index].Dispose(); 
    } 
}; 

while(running)
{
    ThreadPool.QueueUserWorkItem(del);
    Thread.Sleep(1000);
}

This will probably crush performance on whichever machine you're running it on, so you could always implement a simmilar type of loop from several different machines if you have the resources (using processes instead of app domains).

For your third question, give this link a read http://www.albahari.com/threading/

Finally, a stopwatch should be paired with a hit counter to track both duration & unique hits on your server. That should let you perform some analysis after the fact.

1
  • 2
    What possible reason would you have to create separate AppDomains here? That seems completely unnecessary.
    – svick
    Jan 20, 2012 at 8:00
0

Don't bother with threads, if N is reasonably small. To generate N requests per second, use the wall clock time (DateTime.Now). Take the time both before and after the request, then add a Sleep to delay the next request.

For instance, with N=5 (200 ms):

Before request: 12:33:05.014
After request: 12:33:05.077
Sleep(137)
Before request: 12:33:05.214
After request: 12:33:05.271
Sleep(131)

This is not perfect; you may find that Sleep isn't exact. You can keep a running count of deviations (before the X'th requests, the time should be X-1/N later) and adjust the Sleep period accordingly.

Once N becomes too large, you simply create M threads and let each thread generate N/M requests in the same fashion.

4
  • I have to generate very high number of requests. So this cannot be the option because it will drink my memory(4GB RAM) even before 100 threads.
    – King
    Jan 20, 2012 at 14:00
  • I've created 20.000 requests per second from a single thread, in 250K of code. You don't have enough CPU's to run 100 threads anyway (that class of machines doesn't come with 4GB). The next problem would be pushing all those requests out; do you have 10 Gbit/s Ethernet between your load creator and your server? So, you might want to check your actual requirements.
    – MSalters
    Jan 20, 2012 at 14:15
  • To clarify, I have something like 20+ Gbps. So that's not a problem. About the class of machines, What would you be referring to ? number of processors ?
    – King
    Jan 20, 2012 at 15:40
  • @King: to push 100 threads I'd expect a 48 core machine. SGI sells machines with that many cores, for instance, but on those you'd typically get 32GB or more.
    – MSalters
    Jan 20, 2012 at 16:03
0

The easiest way to achieve load testing for any .NET project is to purchase the Ultimate edition of Visual Studio. This comes with an integrated testing tools to help preform all kinds of tests including load tests. Load tests can be preform by creating virtual users either on a single PC or distributed across several for larger numbers of users, there is also a small program that can be installed on the target servers to return additional data for the duration of the test.

This is expensive though, but the ultimate edition comes with a lot of features, so if all were used it would be a more reasonable price.

0

If you purely want to have X threads all hit your resource at the exact same time you can put each thread behind a countdown latch and specify a short wait period between the semaphore checks.

C# has an implementation (http://msdn.microsoft.com/en-us/library/system.threading.countdownevent(VS.100).aspx).

At the same time, if you are stress testing your system you might actually want to check for race conditions as well, in which case you would want to setup thread sleep periods on each thread that osculate over time with randomized frequency and peaks/furloughs.

Similarly you may not actually want to just rapidly send multiple requests, you might have better success at putting your server into a bad state/testing its real world performance by setting up a smaller number of threads that spend more time in consuming and sending messages back and forth over the socket, as your server likely is going to need to spin up its own threads to handle slow ongoing messages.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.