3

I'm writing an application on x86_64 CentOS 7 that functions as a server, managing potentially thousands of devices. Because my application doesn't do so well when running through the connection process (various application-layer handshaking, DB operations, etc.) and it's conceivable that thousands of devices could come online simultaneously, I want to try rate-limiting connection accepts (as a possible "cure at source" approach to my scale problems).

In brief, incoming connections would be "rejected" in some fashion while the application-handshaking process for n devices is already underway (be this an n per T time limit, or a n concurrent handshakes limit).

Sadly:

I hear that the overhead of connect()/accept() is not high (particularly when compared against actual I/O), so I'm inclined to simply shut down my listen socket when I'm already handling (say) 100 reconnections, then re-open/re-listen() it when the coast is clear.

But this somehow feels like the wrong way to go about matters.

How would you approach this problem, given the above constraints?

  • 2
    Just a thought, but have you considered deleting the fd of the listening socket from the epoll instance when you have reached the number of connections you want to service and then adding it back when you are able to handle more? – andy mango Jun 6 '17 at 18:48
  • Does the protocol the clients are using once connected give you a way to say "I'm at capacity; try again in x seconds" with x being a random value as a way to smooth out the connection rate? (In other words, let the server gracefully disconnect and manage the backoff.) – Blrfl Jun 6 '17 at 19:17
  • @Blrfl: No. Though all reconnection attempts will backoff (according to a fixed interval sequence) – Lightness Races in Orbit Jun 6 '17 at 19:55
  • @andymango: FWIW I ended up doing that, and it wasn't as big an architectural problem as I imagined – Lightness Races in Orbit Jul 7 '17 at 13:40
0

leave incoming connections in the backlog, pending an artificially delayed accept()

Artificially delaying the actual detection makes more sense, by temporarily removing the descriptor from epoll's set.

andy mango suggested this in the comments, and it's what I ultimately did (which was architecturally simpler than expected).

That doesn't really answer the question of whether closing and re-opening a socket in quick succession is "a horrible idea", but it does mean I no longer have to wonder.

  • Let me see if I understood correctly: You asked a question, got an answer as a comment, posted that answer as yourself (adding credits) and marked as correct answer? – Machado Jul 7 '17 at 13:48
  • @Machado: That is correct. Posting the final solution used is correct behaviour, as is accepting it if it was the solution used. If Andy had written an answer I would have upvoted it (and probably accepted it too). This is another example of why people break the model when they write answers in comments. But I get no "points" for my actions here, so is there something I can do for you? – Lightness Races in Orbit Jul 7 '17 at 13:49
  • @Lightess, nothing more, thank you. No harm done, hence the reason I wrote "adding credits" on the comment. No innuendos intended. – Machado Jul 7 '17 at 17:18
  • @Machado: Okay great – Lightness Races in Orbit Jul 7 '17 at 17:20
-1

If I understood correctly you made a custom server and on each accept you create an thread. Then when too many connections are open at the same time you server gets overloaded.

If it is the case you should use a ThreadPool. The accept will not be in the main thread anymore. It is something like that;

class Server 
{
     ServerSocket serverSocket;

     class Worker implements Runnable 
     {
         public void run() 
         {
            while(true) 
            {
                 try
                {
                       Socket s = serverSocket.accept();
                       // your code
                 } catch (Exception e) {
                       e.printStackTrace()
                 }
            }
      }

      public void run()
      {
           serverSocket = new ServerSocket(8081,5000);

           new Thread( new Worker() );
           new Thread( new Worker() );
           new Thread( new Worker() );
           new Thread( new Worker() );
      }
}

That way, the server will be opened on port 8081 will allow 5000 connections waiting for accept before rejecting the connection and will work at most on 4 connections at the same time.

There is a implementation that dynamically opens and closes the worker threads on Java.util.concurrent called ThreadPool.

Most webservers already do that. If you implement your service as an servlet you just have to configure your webserver to the size of the pool you want.

  • No, I'm not using any threads and my application is not written in Java. This question is about POSIX socket handling. But thank you for your time. – Lightness Races in Orbit Jul 7 '17 at 13:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.