I'm going to suggest something a little out of left field: learn Python (or perhaps Ruby, but I have a lot more experience in Python so that's what I'll discuss). And not merely dabble in it, but really get to know it at a deep level.
There are several reasons I suggest this:
Python is an exceptionally well-designed language. While it has a few warts, it has fewer IMHO than many other languages. If you are a budding language designer, it's good to expose yourself to as many good languages as possible.
Python's standard implementation (CPython) is open-source and well-documented, making it easier to understand how the language works under the hood.
Python is compiled to a simple byte code which is easier to understand than assembly and which works the same on all platforms Python runs on. So you'll learn about compilation (since Python does compile your source code to byte code) and interpretation (as this byte code is interpreted in the Python virtual machine).
Python has lots of proposed new features, documented in numbered PEPs (Python Enhancement Proposals). PEPs interesting to read to see how the language designers considered implementing a feature before choosing the way they actually did it. (PEPs that are still under consideration are especially interesting in this regard.)
Python has a mix of features from various programming paradigms, so you will learn about various ways to approach solving problems and have a wider range of tools to consider including in your own language.
Python makes it pretty easy to extend the language in various ways with decorators, metaclasses, import hooks, etc. so you can play with new language features to an extent without actually leaving the language. (As an aside: blocks of code are a first-class objects in Ruby, so you can actually write new control structures such as loops! I get the impression that Ruby programmers don't necessarily consider that extending the language though, it's just how you program in Ruby. But it's pretty cool.)
In Python, you can actually disassemble the bytecode generated by the compiler, or even write your own from scratch and have the interpreter execute it (I have done this myself, and it was mind-bending but fun).
Python has good libraries for parsing. You can parse Python code into an abstract syntax tree and then manipulate it using the AST module. The PyParsing module is useful for parsing arbitrary languages, such as ones you design. You could in theory write your first language compiler in Python if you wanted (and it could generate C, assembly, or even Python output).
This investigative approach could go well with a more formal approach, as you will begin to recognize concepts you've studied in the language you're working with, and vice versa.
Have fun!