After researching modern web servers, I feel that I am pretty ignorant when it comes to server design and concurrent task management by both the operating system and the server. I understand some important basic functionality such as thread/process blocking and that the operating system will context switch from a blocked thread to a ready thread. However, I don't think that I have a good enough intuitive understanding of the overhead costs of various designs under the multiprocessing paradigm.
How exactly can a server, or any program for that matter, which processes requests in parallel across all available cores/processors possibly be slower than a server which does so on only one or a few? One cost I am aware of so far is that reading/writing the amount of memory needed for each parallel task is more costly than for one. I understand this point, but then in which cases does this cost not outweigh the added performance of running a queued task in parallel?
I understand server architecture can be very detailed and complicated, but any fundamental understanding that's offered would be very helpful. This question grew from me not being able to figure out how Node.js, which doesn't max out core utilization, can be faster than Nginx (which I have read does). Both have "asynchronous IO", does node just do it better?