Technical interview and programmer ability [closed]

Question

What I will say might be a tad controversial in nature but I am very disheartened today - and so I will ask this.

I just had an interview with a major tech firm for an internship position, where I was asked a lot of typical algorithm oriented interview questions. Now, given my background, I consider myself to be strong in algorithms (I have also got good grades in graduate level algorithms -stuff involving NP-completeness and beyond (approximation and randomized algorithms), but unfortunately I flunked the interview. I could not think of a very efficient method of solving a string problem in approximately ~10 minutes. Once the interview was over, I had a glass of water, ate a banana and relaxed for a while and tried the problem again. And vola! there is the answer I could arrive in under 5 minutes. And the worst of it all - I was actually on that track and the interviewer did hint about it, but too much pressure cooked me. My entire experience got me thinking about tech interviews. I had some questions and I wanted to pose them in this forum -

1. Is it really possible to judge someone's technical ability in half an hour? Honestly? Or is it just a throw of dice?

2. Do technical interview questions measure problem solving ability? This point is very debatable? As a PhD student I know that Mathematical problem solving involves solving something that you have never heard about before. On the other hand questions like - merging two linked lists in sorted order, or printing all the elements of a binary tree in the kth level become "mere exercises" once someone has seen the solution or solved the problem beforehand?

3. Do people who come out with flying colors in these interview go on to become great programmers? Do they go on and design a sleek game engines, graphics libraries, write fast fork-join frameworks? Is there any evidence to point to a positive co-relation between doing well in technical interviews and actual programming ability? Or are these interviews more geared towards finding "getting things done" type of person (Spolsky)?

I can bet that lots of academics publishing ground breaking ideas in - ICML, VLDB, Mobicom - will flunk these interviews. But I can assure you that they are some of the smartest people you will find on this planet.

I am mainly in academia (grad student) - so I will greatly appreciate some perceptive from someone in the other side of the fence. Someone who actually conducts these interviews?

[Ok everyone. Thanks for all the nice and thoughtful responses. Since I do not want to ask another question, I will ask you to answer this question for me.

Suppose candidate X has a good public portfolio of works where he has contributed to some known open source project where - you can actually go and verify his patches, verify the bugs he has closed and take a look at the designs he has created. In that case, the question is how much weightage are you willing to give to his publicly available/verifiable work versus how well he does in answering some very contrived binary tree interview question in under 15 minutes?]

Answer 1

Keep in mind that...

1. The main goal of an interview process is not to arrive at absolute truth about each interviewees personal ability but to select few candidates from a pool of many.
2. It is MUCH, MUCH more expensive to hire a bad developer than it is to pass on a good one.

So a lot of times when interviewing, imperfect shortcuts (such as technical quizzes) are taken because as you mentioned, there's no perfect process for being able to judge a person in 30 minutes. But since most don't have the luxury of working with you side-by-side, you should just accept that it is what it is, an odds game that everyone plays.

Answer 2

Is it really possible to judge someone's technical ability in half and hour?

Not exactly. It is possible to weed out the folks who can't program at all, and those who can't explain the stuff on their CV. Beyond that I'm usually just trying to gauge general intelligence and interest in the field. It's hard to comment on your situation further without knowing the problem you were asked to solve.

Do technical interview questions measure problem solving ability?

That's not the goal of my technical questions. Instead I am trying to discover whether the candidate has a grasp of the fundamentals of computer science. To gauge problem-solving ability, I ask a candidate to tell about an interesting problem they worked on.

Do people who come out with flying colors in these interview go on to become great programmers? Or are these interviews more geared towards finding "getting things done" type of person (Spolsky)?

That's about it. Greatness is rare. I'm happy to find someone competent.

Answer 3

1. Yes. Yes, honestly. This is not to say that 30 minutes is enough time to get a complete picture of what a person is capable of. The interviewer's job is to do there best to get a feel for who the interviewee is and what they can do. Finding the people that put something on their resume that they didn't really know is straight forward. Finding out exactly what the interviewee can do, including all their skills and strengths is not.

2. Yes. They let you see how the interviewee thinks. Getting the correct answer is not always necessary. Seeing someone deal with a problem they struggle with is just as beneficial as seeing someone answer a problem correctly without much trouble. I always ask a question that I think the interviewee might not be able to answer.

3. I don't know of any empirical proof that says technical interviews find the best candidates, but it is the best thing I have tried. Different people are good at different things and getting technical answers correct is not the only thing that decides if someone get hired. Being a good fit for the company is also very important.

Based on your question, I think the biggest thing you need to understand is that the days clearly defined feedback like a letter grade are over. Getting the right answer is only part of the result. Writing good code, communicating your ideas and listening to others is just as important. However, these areas are more subjective and need to be treated as such.

Just like how you solved the problem, take a step back and clear your head. A lot goes into an interview and the interviewer knows that it is not a perfect system. They are trying to do the best they can for the company, just like you are trying to do the best for you.

Answer 4

In a half hour you can judge a person's technical ability is at least as good as x. That is, their ability might be higher than they demonstrated, but you can be fairly sure it's not worse. If x is higher than your requirements, you hire them. Yes, it's unfair if you demonstrate below your actual ability. All I can say is get better at demonstrating.

If someone rattled off a practically memorized answer, I would give them a harder problem. In academia memorization is an important skill. In a job, it's mostly redundant when you have google right there.

Seeing how most great programmers are employed, it's safe to assume they passed a job interview at some point. No, you can't tell if someone will be a great programmer from a half hour interview. Fortunately, most companies don't need great programmers. They get along just fine with solid, reliable programmers who aren't afraid to tackle tasks just outside their comfort zone.

The reason ground-breaking thinkers might flunk job interviews is that opportunities to solve flashy, ground-breaking problems are very rare, and you'll only be trusted to solve them after you've proven you can solve mundane, yet difficult problems your colleagues can't, like why the software crashes when it gets above 16% load.

Answer 5

Very few people know whether their interview techniques really work, because they never learn more about those who failed. But when nothing disproves their hypothesis they take it as confirmation that they are true. So there is a definite element of throwing the dice.

If the economic theories saying markets are rational and efficient were true, then companies would look to hire people that were bad an interviewing in ways that would not make them bad at their job. So someone who wore an ill-fitting suit, sweated, appeared very nervous, and had trouble doing technical problems under the pressure of an interview - if there was outside evidence they could otherwise do technical problems and the pressures of the job were not like those of an interview (as they generally are not) - would represent a company's best chance of getting a candidate better than it could otherwise attract and it would act accordingly. (Think Money-ball for the corporate world). Eventually interview biases would vanish.

In practice this goes against human nature. Interviewers like confident extroverts, no matter what the position, and even if the technical interview provides worse information than the resume and references (as perhaps is true in your case) they will take it into account.

As to your questions:

1. Somewhat. The tests have a poor signal to noise ratio but can be tuned to weed out the completely unqualified at the expense of the qualified-but-bad-at-technical-interviews.

2. See #1. Part of the noise comes from variation in whether people have seen the problem before.

3. See #1. There is probably a tiny positive correlation between these tests and becoming a great programmer.

My advice - Study some sample technical questions to get a basic familiarity. Remember it's a bit of a roll of the dice, so during the interview try to be confident. If you can explain what you are thinking without distracting yourself from actually thinking then do so. If it's really going badly, admit you get nervous at interviews and let them know it seems like something you normally wouldn't have trouble with - possibly giving an example of a similar but harder problem you did solve.