I had a old woodworking teacher as a kid who wanted me to remember to make sure that I didn't leave a mess with varnish on my completed projects. He gave me a silly sounding name to remind me to clean up the varnish before it dried, saying that if he called it something nice I'd forget it, but a silly name I'd remember for ever. He was right, and to this day I associate the word "snotters" with drips of varnish left to dry and make a woodworking project look ugly.
There is power in naming things. We know this in how we sometimes agonize over how we should name our classes and variables in code, and how getting names right helps to define the content of our methods. Likewise I am sure that this is the real power in using an analogy to describe a concept as ephemeral as the potential for a design problem in working code. Agile developers like to get it to work, and then refactor to get it to work better, however a sensible approach needs to be employed to ensure you are refactoring for the right reasons. As Yannis has correctly stated (and I'll paraphrase), that which seems bad to one person may not necessarily be considered bad by another, and thus the potential for a problem does not logically imply the actual existence of a problem.
Perhaps the analogy isn't quite accurate. Maybe a touch, sound, taste, or visual analogy would be more direct. However the problem with applying a non-olfactory analogy is that it is hard to come up with a good short name to describe code that might end up bothering you in some subtle way, while the word smell implies a subtlety that might have otherwise been overlooked had it not been for some factor that brought it momentarily to your attention. What I like about using the word smell is that it offers a way to describe that something may need to be looked at for further tuning, while not necessarily implying that there is anything wrong... like a good and stinky blue cheese might smell awful yet still taste nice. It's subtle - the phrase... perhaps less so the cheese... or this analogy!
Possibly one of the reasons that code smell seems to be treated like bad air on Programmers.SE (yes, pun intended) is that many questions tend to ask "is this a smell", yet without providing enough context. Thus we end up with a pretty useless series of questions that resemble the following:
Q: Could it indicate a problem?
Q: Does it indicate a problem?
A: How will we know without reading all of your code?
Perhaps the underlying issue with people not liking the term "code smell", or perhaps more precisely the reason why many people misuse the phrase, is simply because their definition is being taken from a purists perspective, rather than pragmatically, or perhaps they simply misunderstand what the analogy of a code smell is really meant to represent. Based on the C2-wiki's Code Smell definition it appears that the meaning of the phrase remains sound, and it's usage continues to be applicable, while my now lamented woodwork teacher's wisdom also remains sound, because like Snotters, the phrase Code Smell is hard to forget and will likely continue to be used - and possibly abused - for a long time to come.