2

I was wondering if somebody has done this, and to what success compared to the classic method of getting them to solve a problem as you watch on?

Any pointers on how to carry this out in an interview situation?

I was thinking about doing this for mid/senior candidates which should definitely have the technical knowledge to do a pull request. I was thinking it may be a good character test, if they give a full fleshed out and kind response, as apposed to "NO!". You may really see how they will get on in a team.

Edit:

The way I was thinking about doing this, was to send a candidate a small piece of code before the test. They could read through it, ensure they understand what it does and any technologies it uses (I hate interviews that confuse peoples ability to program with how well they know an API).

Then when the candidate comes to the interview you give them the pull request, using Jira to do a git pull request. Since the web interface just shows you what has changed and the ability to view the raw before and after I do not think this would be large barrier to entry.

My core theory of this was to try to remove the memory game test/pressure cooker a lot of interviews can be and try to let somebody show how they would work on a daily basis with you.

To summarise, I want to know if any one is currently doing/done something similar and what outcomes they see. Also any pain points that became evident when you started this process.

Aside:

Thank to everyone who has answered so far they are helpful and very valid but my original intent with this idea was to stop interviews looking for cookie cutter programmers and to let raw talent shin. I wanted to test this theory here with fellow programmers that would possibly go through such a process.

closed as unclear what you're asking by Robert Harvey, gnat, user22815 Dec 9 '15 at 5:09

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • A pull in what system? There are more than a few software systems out there that could use the term "pull" so your question may be seen as BS in some places without more context. – JB King Dec 9 '15 at 18:33
  • Just to add a bit here, I would expect to get paid to submit a change set into your repository. I'd be happy to demonstrate on an example code base, but it would need to be obviously contrived. You could just limit yourself to people with active GitHub or Bitbucket account, but... Well, there's a very nice answer below telling you why you shouldn't. – RubberDuck Dec 11 '15 at 0:43
10

When I was 6 and my brother was 11, he made me eat a ball of feathers. It wasn't easy. I choked it down with a lot of dry gagging and a little wet vomit. Partially I did it because I was afraid of my brother. But mostly I did it because it proved I was harder than him. He was this vicious, brilliant, beautiful boy. I was a puny, weak, and ugly gerbil. Regardless, it was I who could do the impossible. I suffered through his torments without the tears he craved because I knew it made me stronger than him. I exposed his inherent weakness.

My point, as it relates to your question, is that I have a set of values that are important to me. It sounds as if you value pull requests like I value obstinate individualism, and that's great. The thing is, I've since learned that I miss out on the textures and insights of many interesting people if I insist that they share all of my values. If you insist on a live pull request during an interview, you'll likely vote many capable (and interesting) people off the island on the basis of that one task.

I know, I know, a pull request should be a bare minimum for a professional at the level you're describing. Right? I agree in regards to short-term opportunities, but with the long-term... I throw out those restrictions. I don't want to miss that hidden gem. What about the brilliant physicist who didn't get tenure and is looking at new opportunities? She's an amazing programmer, has a completely original way of looking at the world, and would elevate your team. But... she can't submit a pull request to save her life. What about the old IBM iSeries developer? He has a nuanced and subtle understanding of software after decades in the business. He's experimented with a ton of technology outside of work -- Haskell, npm, Docker, Haxe -- he loves it! He's a great poker player because he can read people. This is indispensable for intuiting what is really needed in his software. But... he's never submitted a pull request. To make matters worse, he gets a little nervous and sweaty in interviews. I don't want to miss the opportunity to work with these people!

Also, be careful of "get on in a team," "get along on a personal level," and "interact well with team members." Don't focus on that too much early (but don't sack good sense). Yes, you want to avoid the lunatics, but "getting on" is often one of those "cultural fit" requirements, and those are often code for people-who-are-just-like-me. Instead, focus on people who are wicked smart and have unique insights into problems, maybe even if they're not your problems. That's what an interview is for me; it's a flexible platform for promoting smarts. I want it to be loose enough so that your peculiar brilliance shines through on its own terms. Our best hire in the last 10 years came to us as an intern from a woman's correctional facility. I suppose that's a story for another time.

  • 4
    Do you have a blog? Please get a blog! This is a wonderful piece of writing that is both insightful and captivating :) – amon Dec 9 '15 at 11:52
4

I'd say no to this. Reviewing anything but a trivial pull request (spelling fixes) requires an understanding of the code base. If you do not know the codebase, you may not be aware that a certain (obviously good) change might have some major effects somewhere else. Even veterans to a codebase can miss the repercussions of 'simple' change.

What you want to measure is how they will get along with the team. This requires them to get along on a personal level, as well as them being flexible enough to adjust to the flow of the team. To figure that out, talking to them works pretty well.

3

I haven’t done this, but I think it would be a good idea, provided you’re willing to field questions from the candidate about the codebase and what the change is intended to accomplish. They can’t be expected to immediately understand a foreign system, but they should demonstrate how they would go about understanding it.

I would expect an experienced developer to be able to give a high-level technical critique, e.g.:

  • Are the data structures and algorithms appropriate for the problem?

  • What are some possible sources of errors?

  • How could this code be made clearer?

That is, you’re not looking for “this whitespace is wrong”, you’re looking for “an RB-tree would be better than a hash table here, because X” or “this comment could be improved, by mentioning Y”. You’re also looking for the candidate’s ability to talk intelligently about code, and interact well with team members.

Not the answer you're looking for? Browse other questions tagged or ask your own question.