3

So we're having an internal debate on whether the time restriction that we have in place for our technical test a good thing. I thought I would open it out to the community. So what are your thoughts on this restriction, do people accept it as standard practice or is it an artificial restriction?

A little background, for all our positions, we expect candidates to complete a coding exercise and we set a time limit which we think is reasonable to at least layout the design (we don't expect a fully working solution), and it is this time limit which is under debate. The current time limit is 2 hours (which I frankly think is generous.)

Improve this question
6
  • What kind of time limit are you talking about? Hours? Days? Weeks?
    – Oded
    Commented Mar 8, 2012 at 11:55
  • @Oded, updated the question...
    – Nim
    Commented Mar 8, 2012 at 12:00
  • When you state you "we don't expect a fully working solution" are you telling that to the coder, since if you're not, or only giving that information if asked, it will effect the results.
    – blunders
    Commented Mar 8, 2012 at 12:42
  • Bear in mind that you always have the option of not mentioning the time limit and just letting someone get on with it. Whether they went over or stayed under the limit can simply be a factor in assessing what they came up with.
    – Marjan Venema
    Commented Mar 8, 2012 at 13:51
  • @blunders, as I mentioned in a comment below, we state that in the spec, "we don't expect a fully working solution", just your thoughts...
    – Nim
    Commented Mar 8, 2012 at 16:00

6 Answers 6

Reset to default
5

One way you might evaluate the legitimacy of the time limit and the questions involved is to give the same exercise to a sampling of staff that are already employees. Otherwise you're literally guessing based on a preconceived idea about what someone "should" be able to do.

By doing small technical exercises in a short time you are evaluating a narrow indicator of performance on a very artificial situation. The results of such a test have to be taken in context with other factors. Probably not a good idea to use the results as a strict "go / no go" filter.

Improve this answer
1
  • Funnily enough, all of the folks here have passed that test (aside from one or two founders!) ;)
    – Nim
    Commented Mar 8, 2012 at 15:59
3

When it comes to interviews it makes sense to have time limits - otherwise you can end up interviewing all the time...

For coding exercises, the time limit should reflect expectations from the candidate - if you believe the exercise can't be finished in the allotted time you should communicate this to the candidate (otherwise they might believe they are expected to finish the whole thing and get discouraged and not do their best). You should also communicate what the candidate is expected to accomplish.

Even though this is an artificial limit, it needs to be in place - candidates know how long they have to spend on the exercise and it also tests how they self manage their time and how they self organize.

Improve this answer
1
  • We do communicate exactly that (i.e. we don't expect a fully working solution, pseudocode with some descriptions is sufficient), we just want to see their thinking, hence why I am in favour of the time limit.
    – Nim
    Commented Mar 8, 2012 at 12:05
1

You seem to have the process in place for some times. I'd suggest to analyze its results. How do the candidates behave? How did the candidates you ended up by hiring behaved at the test? Were they satisfying? Then you know if the current process is too lenient or not, and what are the problems it doesn't detect. (Sadly it is difficult to know if and why it rejected persons who would have been good hires).

Improve this answer
4
  • That's the counter argument to the time limit, by placing this artificial limit, are we also rejecting good but slow candidates.. I guess there is no real win...
    – Nim
    Commented Mar 8, 2012 at 16:02
  • 1
    @Nim: at some point, isn't slowness disqualifying?
    – kevin cline
    Commented Mar 8, 2012 at 16:31
  • @kevincline, I totally agree, I just wanted to canvass some opinions to see what the general thinking is. The 2 hour time limit is more than generous for anyone of any experience to come up with some sort of design. If they cannot, that in itself is a no-no for me (implies they cannot really think on their feet/adapt/react quickly...)
    – Nim
    Commented Mar 8, 2012 at 16:39
  • Knowing if an hiring process is too strict is difficult. But as long as it doesn't prevent to hire timely and isn't too resource consuming, that is not really a problem from the company POV -- the homogeneity of the hired people has both advantages and drawbacks. But if too many employers are biased in the same way, it is a problem for those looking for a job.
    – AProgrammer
    Commented Mar 8, 2012 at 16:50
1

Time limits give you an out. A competent dev should be able to come up with at least a first approximation solution, even if its just pseudocode or interfaces. Someone who is not competent will immediately start coding, and will tangle themselves up.

I haven't done that many interviews (< 50 ) in my career so far, but I have found that candidates who finish within the time period are generally competent, whereas those who go over the time-limit are either detail-or-analysis-obsessed or incompetent, and further questions about their design can weed this out.

Improve this answer
3
  • detail-or-analysis-obsessed should not be such a bad thing, should it?
    – Andrei G
    Commented Mar 8, 2012 at 13:06
  • It is when it leads to analysis paralysis. If you fail to deliver a project on time because you spent too much time in analysis, it's as much a failure as if you spent too little time in analysis. Realistically, the sort of problems set during interviews are things like 'Count the number of words in this file on the filesystem', not 'Write us a fully-functional Document management system'. In my experience, anyway :)
    – mcfinnigan
    Commented Mar 8, 2012 at 13:11
  • I like "analysis paralysis"... :)
    – Nim
    Commented Mar 8, 2012 at 16:03
0

I think, in general they are a good idea. It allows you to draw on another comparison between two candidates.

For example, if Candidate A completes the task in an hour and scores 80%, Candidate B completes the task in one hour and half and scores 80%. Naturally you would go for Candidate A.

Every other variants on this depends on your preference. Do you want someone that takes longer to do things but also does them very well. Or someone that does it quickly and does good enough.

It will also show you how the candidate reacts to a pressurised time limit (2 hours is far too long). If they rush, their code will be sh*te but others will take their time, even knowing they won't finish within their time limit.

It's entirely up to you essentially.

Improve this answer
0

There is no project that I know of that has been running indefinitely, such a thing is an utopia.

Since the prospective hire is going to be under permanent stress and his tasks will always be o a time limit, it comes naturally that he should be subjected to a test with a time limit.

But, be careful, him being able to finish the test under your time limit does not automatically make him competent (or the other way around) if you do not set the limit properly.

Improve this answer
3
  • 1
    Some U.S. federal government software projects approach indefinitely. Utopia is not a word that comes to mind when thinking about those projects.
    – Gilbert Le Blanc
    Commented Mar 8, 2012 at 13:51
  • While I do understand the concept of very large and long lasting projects, I am more inclined to think that the projects you are talking about are actually successive projects either trying to achieve the same goal or even with different goals. But I am genuinely curious as to what projects you are talking about.
    – Andrei G
    Commented Mar 8, 2012 at 14:18
  • Modernization of the IRS systems. The project has lasted so long, CTG built a building next to the IRS offices in New Carrollton. Modernization of the FBI case management system. I'm sure Google will provide details.
    – Gilbert Le Blanc
    Commented Mar 8, 2012 at 14:28

Not the answer you're looking for? Browse other questions tagged or ask your own question.