It is mainly about memory (as Michael Borgwardt said) with a bit of JIT inefficiency added in.
One thing not mentioned is the cache - to use the cache fully, you need your data to be laid out contiguously (ie all together). Now with a GC system, memory is allocated on the GC heap, which is quick, but as memory gets used the GC will kick in regularly and remove blocks that are no longer used and then compact the remaining together. Now apart from the obvious slowness of moving those used blocks together, this means that data you're using may not be stuck together. If you have an array of 1000 elements, unless you allocated them all at once (and then updated their contents rather than deleting and creating new ones- that will be created at the end of the heap) these will become scattered all over the heap, thus requiring several memory hits to read them all into the CPU cache. A C/C++ app will most likely allocate the memory for these elements and then you update the blocks with the data. (ok, there are data structures like a list that behave more like the GC memory allocations, but people know these are slower than vectors).
You can see this in operation simply by replacing any StringBuilder objects with String... Stringbuilders work by pre-allocating memory and filling it, and is a known performance trick for java/.NET systems.
Don't forget that the 'delete old and allocate new copies' paradigm is very heavily used in Java/C#, simply because people are told that memory allocations are really fast due to the GC, and so the scattered memory model gets used everywhere (except for stringbuilders, of course) so all your libraries tend to be wasteful of memory and use a lot of it, none of which gets the benefit of contiguity. Blame the hype around GC for this - they told you memory was free, lol.
The GC itself is obviously another perf hit - when it runs, it not only has to sweep through the heap, but it also has to free all unused blocks, and then it has to run any finalisers (though this used to be done separately the next time round with the app halted)(I don't know if it still is such a perf hit, but all docs I read say only use finalisers if really necessary) and then it has to move those blocks into position so the heap is compacted, and update the reference to the new location of the block. As you can see, its a lot of work!
Perf hits for C++ memory comes down to memory allocations - when you need a new block, you have to walk the heap looking for the next free space that is big enough, with a heavily fragmented heap, this is not nearly as fast as a GC's 'just allocate another block on the end' but I think it is not as slow as all the work the GC compaction does, and can be mitigated by using multiple fixed-sized block heaps (otherwise known as memory pools).
There's more... like loading assemblies out of the GAC that requires security checking, probe paths (turn on stracesxstrace and just look at what it's getting up to!) and general other overengineering that seems to be much more popular with java/.net than C/C++.