Do not mistake Node.js for a web development framework. It can be used that way, but so can Ruby without Rails. That doesn't mean you should.
Node.js is simply an event-driven library, written in Javascript. That is, it runs a script in a single thread, but using a system of queues and callbacks to free up that thread for other processing whenever it is doing something which takes time but not resources.
This replaces thread-based libraries, which tend to move the processor from thread to thread when they are all trying to work. This kind of context switching is costly so, generally, event-driven will be faster.
There are a couple of caveats to that though. Event-driven paradigms do not run over multiple processors so, if the load is fairly low, a thread-driven paradigm can be faster by sharing its work across cores (this is not an assumption you can make for certain with all frameworks though).
[Edit, ref Raynos's comment] It appears that the above is easily resolved by running a load-balanced cluster of node.js servers -- one for each core.
And code written to use event-driven paradigms must free up the thread whenever it can, otherwise all other requests will queue up behind it, making it considerably slower than a thread-based paradigm. Luckily most of the work there has been done for you and there are plenty of web development frameworks which sit on top of Node.js to make your life easier.
Pick the right one of those for your needs and yes, you will be able to use Javascript throughout your development stack.
some server-side scripting language (like c#, php)
so that you only need to know javascript for both the client and server side