For example, in case of GDB it is possible to debug itself by itself. I have not been able to find any JVM that can debug itself or even allows debugging on source level. I was just wondering if someone can suggest on how the systems are made self debuggable. If you have references, please paste it here. I wanted to read on it in detail.
This has nothing to do with GDB being able to debug itself. GDB is a debugger for C. It can debug any C program. GDB also happens to be written written in C, therefore you can debug GDB with GDB.
The JVM Debug Interface is an interface for debugging Java. You can debug any Java program. Most JVMs aren't written in Java, and even if they are written in Java, they don't run inside of themselves.
There is however a JVM, which is written in Java and runs inside of itself: the Maxine Research JVM, and it can debug itself.