I recently asked a question about tests in game development - this is BTW how I knew about this one. The answers there pointed some curious, specific disadvantages:
- It is costly to do when your code should be highly coupled.
- It is difficult to do when you have to be aware of the various hardware platforms, when you should analyze output to the user and the code result only makes sense in a broader context.
- UI and UX testing is very hard.
- And notably, automated tests can be more expensive and less effective than a bunch of very low-cost (or free) beta testers.
The 4th point makes me remember of some experience of mine. I worked on a very lean, XP-oriented, Scrum managed company where unit tests were highly recommended. However, in its path to a leaner, less bureaucratic style, the company just neglected the construction of a QA team - we had no testers. So frequently the customers found trivial bugs using some systems, even with test coverage of >95%. So I would add another point:
- Automated tests may make you feel that QA and testing are not important.
Also, I was thinking those days about documentation and cogitated a hypothesis that may be valid (to a lesser extend) to tests two. I just felt that code evolves so quickly that it is pretty hard to make documentation that follows such a velocity, so it is more valuable to spend time making code readable than writing heavy, easily outdated documentation. (Of course, this does not apply to APIs, but only to internal implementation.) The test suffers a bit from the same problem: it may be too slow to write when compared with the tested code. OTOH, it is a lesser problem because the tests warn they are outdated, while your documentation will stay silent as long as you do not reread it very, very carefully.
Finally, a problem I find sometimes: automated testing may depend upon tools, and those tools may be poorly written. I started a project using XUL some time ago and, man, this is just painful to write unit tests for such platform. I started another application using Objective-C, Cocoa and Xcode 3 and the testing model on it was basically a bunch of workarounds.
I have other experiences about disadvantages of automated testing, but most of them are listed in other answers. Nonetheless, I am a vehement advocate of automated testing. This saved an awful lot of work and headache and I always recommend it by default. I judge those disadvantages are just mere details when compared to the benefices of automated testing. (It is important to always proclaim your faith after you comment heresies to avoid the auto da fé.)