I think it depnds on the interview situation and the questions. I have used both techniques.

Why might I want to not ask follow-up questions? When I am trying to find out the person's response to stress. I have interviewed people for some jobs that were in highly stressful environments and how well they could handle stress was a critical factor in our evaluations, so we asked some extremely difficult questions that no one could answer without some stress.

When I am trying to find out their technical knowledge, I ask follow-up questions that may contains hints as to what I am looking for. Contrary to the thought of the manager who said you have to ask everyone the same questions to be fair, I believe this is fair as long as several conditions are met. First everyone gets asked the same base question. Second, you should not ask follow up questions to help only one person. If you have let others flounder with no help, you need to let all flounder with no help. Second, you should compare candidates' performance on the question, not only in terms of their final answer but in terms of how hard it was to drag it out of them. This process still treats everyone fairly.