I'm studying asynchronous IO, concurrent models for IO and how things works on windows, linux and most used web frameworks.
I'm struggling on understanding why single-threaded event loops like the one used by node.js or ngnix, in case of handling requests that wants to make an IO operation (let's say an HTTP request to get data from another service), uses a dedicated thread for each IO operation, instead of using just one thread and Epoll to handle all of them.
Let me explain better:
- Every time I read about Epoll I read examples or tutorial from a server perspective: I'm a server -> I hate thread's context switch -> Epoll let me handle a lot of incoming requests with just one thread. Nice.
- So I build a server tech like node.js: single-threaded event loop and non-blocking IO operations. How can I make non-blocking IO operations? I execute them on dedicated IO-threads and each of this thread will block until the IO operation is completed (letting the main event thread free to handle other requests).
- It means that if I receive 100 requests, and for each of them I have to make an IO operation, I could end up using 100 threads or the maximum number of threads available in the thread pool leaving lot of IO operation waiting for a free thread.
So what's the point of using Epoll or others multiplexing libs to save on thread count for managing incoming requests if I could end up doing a lot of context switches for managing IO operations needed by each handled request to complete? Why are not these IO operations "batched" to be performed on a single (or few) threads using Epoll?