The way you learn how the computer works at a deeper level is by learning a low-level language like C or, better yet, assembly language. Making your own programming language teaches you about language abstractions, not the inner workings of a computer (unless you use C or assembly language to implement your language).
If you do choose to implement a programming language, I suggest you try implementing a small LISP. LISP is very easy to parse, and will allow you to write an interpreter or compiler quickly, without getting bogged down in detail. Bonus points for writing your LISP in C.
If you're still unconvinced...
Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot. -- Eric Raymond
Build Your Own Lisp
Minimal Implementations of LISP
McCarthy's Original Lisp