14

I think our company can create challenges designed to find software engineer candidates who are:

  • Good at solving problems, not at wowing recruiters.
  • More likely to be afraid to come up to us at a career fair.
  • More likely to be underutilized in their current programming job but are too introverted to do anything about it.

For an example see this article that discusses Facebook hiding an email address in an image using Piet.

I just can't find any studies or hard data on whether this actually works or not.

Improve this question
14
  • I disagree. It's not unusual to have two-line titles on SE websites, and this specific title is very clear as is. Shortening it may make it more confusing.
    – Arseni Mourzenko
    Feb 8, 2012 at 0:53
  • 1
    I can't imagine a serious study. How do you decide whether the ones, which are hired after such challenge are better than those who would have been hired otherwise, or vice versa? Programmers differ, their education changes much over the decades, recruiter change, the challenges change, a usable scoring system is hardly imaginable.
    – user unknown
    Feb 8, 2012 at 3:28
  • 1
    Hi Joe: requests for studies tend to do poorly here: our expertise is not in information retrieval. If this was phrased as, "How effective are programming challenges in the recruiting process?", it'd likely do much better.
    – user8
    Feb 8, 2012 at 4:13
  • 1
    @mattnz: I don't see where your conclusion comes from. You can perform animal test with tobacco smoke with mice. You can measure reaction speed in a simulator after people drank some alcohol. How can we transfer these methods to hiring programmers?
    – user unknown
    Feb 8, 2012 at 4:29
  • 3
    @mattnz, number of deaths per 10,000 persons over a specific period of time, whether due to lung cancer or to road accidents, is a (more or less) objectively measurable quantity. Goodness of a developer, or success of a SW project, are not even well defined terms.
    – Péter Török
    Feb 8, 2012 at 9:27

3 Answers 3

Reset to default
7

Like any tool, they can be extremely helpful, or extremely dangerous. A power drill will make your life so much easier - until you drill through the top of your hand and land yourself in the ER. The same is true with programming challenges in recruiting.

The Good: This may be an effective way to detect someone who, on paper, might not be all that compelling as a programmer. The one with a degree in something that has very little to do with what people normally consider "programming" related fields - Biology, Political Science, Art History...

If they blow through your challenges, then great. They learned programming, somehow, and it's apparently stuck. If they get bogged down, their application may really just be something that slipped through HR.

The Bad: A poorly written programming challenge doesn't actually assess programming skill. It tests puzzle solving via programming skill. The problem is the later is a two variable question - are you good at puzzle solving, and can you do said puzzle solving via code. It's possible to have a perfectly talented programmer who utterly fails at the puzzle solving part.

Most programming challenges I've seen also fail at detecting people who are close to what you want, depending on how it's written.

There's ways to mitigate both of those. For the latter, I'd consider accepting "partial credit" in the form of solutions that don't seem to quite be getting there, "Here's how I would solve this..." etc. if you're genuinely looking for problem solvers. After all, very few people code all alone, and if their answer would have been right if they could ask a senior colleague "Hey Jim, do you know a good way to implement X?", that may very well be someone you want on your team.

The former is somewhat harder, because the burden for that is on you. Pick puzzles/problems/challenges that matter. If no one in your group has ever come up against anything even remotely resembling the Traveling Salesman problem in their work, don't make some clever spin on Traveling Salesman the challenge you come up with. That way, if they're failing at the problem solving aspect of "solve the problem and code it", they're at least failing at something that will actually come up, rather than some arbitrary bit of cleverness your team spitballed over lunch.

Improve this answer
1
  • +1. Creating a good programming challenge is a real challenge for the recruiter.
    – Simon Bergot
    Feb 10, 2012 at 12:43
6

Very effective.

... as long as your recruitment process doesn't just contain programming challenges. While I fret and hate doing the technical assessment of any interview, it does act as a simple gauge to filter out idiots. And filtering out idiots is the crux of the recruitment process, so you can spend more of your time on those who are fit for the role.

When interviewing, I deem it very important to see what people say under pressure. If they're incline to spit out a bunch of blatant crap, it's easily identifiable and I'll know this person isn't worth my time.

More likely to be afraid to come up to us at a career fair.

This isn't a bad thing. If your potential candidate isn't willing to bet that he's worth being employed there, do you really want to recruit him anyway?

Improve this answer
1
  • 1
    There may be other reasons why someone would be afraid to come up to them... For instance, some people find it difficult/scary to try to sell, or even just present, themselves. Feelings get in the way. That doesn't mean they wouldn't be brilliant and/or valuable once they get past that initial making contact fase.
    – Supr
    Feb 10, 2012 at 13:06
0

I am assuming you want someone to work as part of a team - as such the better programmer is the person who works better with the existing team members. You want to get a group of people together that can effectively communicate to each other, who actually get along well with each other (they don't have to be friends, but they need good rapport and respect), who are willing to agree and use a common development standards (code and process), who are willing to help their colleges when they deal with a new issue or have a mental block ( four eye theory ). You also need to find a mix of personality types, so if you have a team of introverts that rarely talk, then bringing in a more talkative team member may well improve team dynamics which will make the team more productive. On the other hand, you want to avoid a person with a overly strong personality because they will dominate the team and that will reduce quality and productivity - particularly with introverts in the team.

After you get a person to fit into that mix, then consider technical skill/ability. These too need to complement. Every one has different areas that they are strong at, others they are OK at and some they have no clue. So you need to get a mix of strengths together relevant to the project at hand. Remember that an intermediate coder who works well with a good coder will have the level of their work raised by the stronger person. The weak link in the chain is relationships, not skills (provided the skill is in the team)

Good luck in putting that together.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.