5

I currently have a setup looking like this:

 __________________      _________      ________      ___________
| Front end server |----| Varnish |----| NodeJS |----| C-service |
 ‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾      ‾‾‾‾‾‾‾‾‾      ‾‾‾‾‾‾‾‾      ‾‾‾‾‾‾‾‾‾‾‾

NodeJS and C-service runs on the same virtual server.

The front end server may have hundreds of clients connected - each of these can generate hundreds of concurrent requests to the C-service, so the resulting requests to the C-service will number in the thousands.

From previous experience I know that sockets can only handle up to some limit of waiting connections (backlog). To handle that I implemented a message queue in NodeJS (it also converts HTTP messages into messages that the C-service understands, but that's a minor thing I could be doing in the C-service instead).

Problem:

  • Varnish handles the thousands of requests easily
  • The C-service can keep up with the thousands of requests, as long as they are queued.
  • NodeJS... Not so much. (I tried clustering, but due to other implementation issues this is not an option.)

In other words, I have a NodeJS server that is severely limiting my throughput, but I don't know how else to handle that many concurrect requests.

I've been looking into message brokers such as Kafka/RabbitMQ/ZeroMQ, but they don't seem to handle my basic problem of connecting an HTTP service (Varnish) to an underlying TCP/Unix domain socket with message queuing. I'm not sure they're well suited for synchronous operation either.

Is this the wrong approach? What else could I be doing?

I need the responses to be sent synchronously, but not necessarily in the same order as they are sent from Varnish (I can run multiple instances of the C-service for better CPU utilization in which case sending the responses in order would slow things down).

migrated from serverfault.com Jun 22 '16 at 12:31

This question came from our site for system and network administrators.

  • 1
    Nodejs seem to be run on top of javascript, your answer is there, nodejs cant handle the same load as a pure service coded in c. We cant do much there – yagmoth555 Jun 22 '16 at 10:51
  • I've never actually used nodejs, but from what I've read about it, the way it scales is to do everything asynchronously, in a single thread. If the c service that it's calling into is synchronous, that won't work out very well... – Bwmat Jun 22 '16 at 15:34
  • I would try to run the c service in multiple worker processes, with a message queue between nodejs and them, doing as little processing in nodejs as possible – Bwmat Jun 22 '16 at 15:36
  • By synchronous I mean that I want the client to receive the actual response to their request - not a "queued" response where they have to poll until the answer is ready. – Woodgnome Jun 22 '16 at 15:39
  • I am already using NodeJS in the way you describe (multiple C workers with NodeJS purely queueing items and sending requests to the C-sockets with async communications and sending the response in a callback function) - but with several thousands requests per second even NodeJS can become the bottleneck. – Woodgnome Jun 22 '16 at 15:40
0

Seems I was indeed looking for the wrong tool for the job. Well, maybe message brokers can do the job, but there is an easier solution: HAProxy.

HAProxy has a built in max connections settings for it's backend servers and will queue messages until the backend server can respond. It also supports connecting to Unix domain sockets (since version 1.5).

In the backend server definition add maxconn [limit], for example:

backend backend-server
    server backend 127.0.0.1:8080 maxconn 5

To support thousands of incoming requests it looks like it's necessary to add maxconn in both the global and frontend sections of the config, for example:

global
    maxconn 10000

frontend http-in
    bind *:80
    maxconn 10000
0

On the C side, you might consider using some message library, like 0mq.

Perhaps you could be interested by JSONRPC or some JSON libraries like jansson (probably to be used above 0mq...), since node.js is likely to be JSON friendly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.