It is popular to write documentation in the same file as the code and extract that using a >software to generate documents.
Yes, that is popular - for software libraries and their documentation. I guess you are talking about that kind of docs (and not, for example, user manuals).
And that often results in cumbersome source file.
Well, that is quite debatable. When done with care, IMHO the documentation is exactly where it belongs - in eyesight of the thing it describes, so when I look at a method signature or implementation, I have also to short description of the intensions of that method at hand.
Doesn't it make more sense to write documentation in the same file as tests not in commented lines but in ordinary language using DSL?
I see some problems here:
in your docs, you typically want a general outline of your class, and a description for each public method or property of your class. So the docs follow in structure exactly your classes structure, but your tests have typically a different structure. For a specific method, you will typically have more than one test, for a class you will have a lot tests and for a property you might have no test at all.
when you place the docs not directly above a method, you will have to repeat the complete method signature and function name at the docs place, to make the description readable - so this leads to some form of unwanted code repetition. Placing the doc above the method avoids that.
docs nearby the documented function have the highest chance to be kept up with the code when something is changed. When you put your docs somewhere else, you lower that chance significantly.
So my answer is - no, I don't like that idea (but I like the question, so +1 for that).
What I like, however, is the idea of using tests as part of the documentation (not the full documentation, they are the examples part, but don't describe the general requirements). The documentation could give references to specific tests showing example behaviour, and a doc extraction tool could use that references to create a formatted document where those examples are included (don't know if such a tool exists, but perhaps this would be a nice new feature for one of the available products).