I currently have a setup looking like this:
__________________ _________ ________ ___________ | Front end server |----| Varnish |----| NodeJS |----| C-service | ‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾ ‾‾‾‾‾‾‾‾‾ ‾‾‾‾‾‾‾‾ ‾‾‾‾‾‾‾‾‾‾‾
NodeJS and C-service runs on the same virtual server.
The front end server may have hundreds of clients connected - each of these can generate hundreds of concurrent requests to the C-service, so the resulting requests to the C-service will number in the thousands.
From previous experience I know that sockets can only handle up to some limit of waiting connections (backlog). To handle that I implemented a message queue in NodeJS (it also converts HTTP messages into messages that the C-service understands, but that's a minor thing I could be doing in the C-service instead).
- Varnish handles the thousands of requests easily
- The C-service can keep up with the thousands of requests, as long as they are queued.
- NodeJS... Not so much. (I tried clustering, but due to other implementation issues this is not an option.)
In other words, I have a NodeJS server that is severely limiting my throughput, but I don't know how else to handle that many concurrect requests.
I've been looking into message brokers such as Kafka/RabbitMQ/ZeroMQ, but they don't seem to handle my basic problem of connecting an HTTP service (Varnish) to an underlying TCP/Unix domain socket with message queuing. I'm not sure they're well suited for synchronous operation either.
Is this the wrong approach? What else could I be doing?
I need the responses to be sent synchronously, but not necessarily in the same order as they are sent from Varnish (I can run multiple instances of the C-service for better CPU utilization in which case sending the responses in order would slow things down).