Languages are not usually tested, but implementations are often tested very thoroughly – as very basic infrastructure, they have to work. However, unit testing a language is difficult since we can't test a feature in isolation. There is no way to delimit the “unit”. The different features of a language have rich and complicated interactions, and a valid program required for a test case will necessarily touch on many things that are not the goal of that specific test.
Such tests are integration tests: a whole program is executed and is expected to produce a particular output. Often, the language infrastructure includes a custom test runner to manage this. E.g. the TAP test result format originated with early Perl interpreter test cases.
The main source of new test cases is often regression tests: A user found a problem, and we want to make sure it never has to be fixed again. Some languages are effectively defined by their reference implementation (e.g. Ruby, Perl, Python). The language is whatever that reference implementation does. This makes tests a bit pointless. However, a complete test suite can be used to create compatible alternative implementations.
Many languages are defined by a specification. A specification cannot be tested or verified directly. Usually, the language already exists at least in prototypes or reference implementations before it is formalised. An interesting counter-example is Perl6, where the (informal) specification was translated into a suite of test cases before any implementation existed. This made it possibly to track the progress of potential implementations very visibly. One could see straight away which language features had yet to be implemented. Java also has a canonical test suite, but that is more related to the Java trademark than the Java language specification.
For parsers, fuzz testing may be very valuable. This doesn't really test the language, but makes sure the implementation is well behaved and uses defensive programming. Here, input to the implementation is mutated randomly. Usually, these changes should result in illegal syntax, so what we expect is an error message and a clean exit. If an error within the implementation is triggered, that test input is saved for human review as it likely indicates a possible bug.