Pre-screening coding test - How long is reasonable? [closed]

Question

EDIT

After a good amount of thinking and self-reflection on the topic, I realised that most of the issues I raised in this question was coming only from a personal, rather than a professional perspective. Hence the moderators put this question on hold because of the highly personal, subjective nature of the problem I tried to talk about. I was thinking about rephrasing the question but I could not really find a possible way to manifest the question in more objective way so it can be the subject of a discussion where answers can be back up with some sort of evidence or references.

For the sake of those who are still interested, I am trying to give a summary of the discussion emerged from this question:

• a 4 hours pre-interview, offsite programming test is not usual but
• many people pointed it out that for some companies you would interview for much-much longer than that all together
• it is our personal decision if we take a test or not, and we can evaluate this based on our circumstances and the perceived benefits of getting hired for the company
• all companies are different, as people are, and it can be perfectly reasonable for a company to employ a longer pre-interview offsite test, if that is what fits their needs or circumstances

I wanted my original question to be about how reasonable to expect 4 hours from me, and how ethical to give out a problem so the solution (not the code, but the design) can be possibly used for the company. As I can now see both of these questions can only (at best) be explored in a forum discussion, rather than using a question-answer type community tool like stackexchange.

However, I found all your answers valuable and thanks for sharing.

ORIGINAL POST

I am interviewing for several positions, and most of them include a pre-screening phase where I have to submit a coding test before the telephone interview or the onsite interview would take place. I have pretty much got used to this idea, and find it quite reasonable that companies expect me to do this so they can check what type of work I can produce on my own.

Generally, my experience is that these type of coding exercises are mostly small programming tasks. Do some logic, maybe implement a small algorithm, open a file and read/write data, stuff like that. Even the most simple task can be implemented with nice separation of logic, testable components, etc, to see how the candidate is coding, generally how well he is prepared for the type of job a company want to fill in.

Recently I came across a company who sent me a coding test with a whole page long description of their exercise, asking me to solve a real life problem of their business (I don't want to say specifics to protect the company, but the test was pretty much about what they do). They described a pretty complex system to implement, included real data, and in the end they concluded that the coding test should not take more than 4 hours.

Is it reasonable from a company to expect me to spend 4 hours working on their dummy assignment in my free time, even before they would say hi to me? (the recruiter sent me the coding test)

Don't get me wrong, I am motivated to find a new job and new challenges, but most companies expect me to spend maximum 1-2 hours on a task like that, and such tasks has always been far less complicated.

What I came up as a conclusion with this company is that either:

1) My motivation is not good and probably they are looking for someone else

2) They do not respect their future employees to expect such a long coding tests to do even without saying hi to them

3) They just want to give out one of the problems they work on and see if there is an enthusiastic young fella who would solve it for them for free (again, don't get me wrong I am not a conspiracy theorist but I have heard such stories ...)

How much do you think is reasonable for a company to expect candidates spend time on their dummy coding tests without talking to them? What is your experience generally?

Answer 1

Let me take the company's side for a moment, since the other answers haven't so far. It would be nearly impossible to build a usable code base out of a conglomeration of 4-hour coding test submissions from people whose qualifications are completely unknown. Creating a detailed enough specification, vetting the responses, and integrating it with the rest of your code would take longer than 4 hours. Not to mention most useful enterprise-level software projects require thousands of man hours. The thought of building a business on splitting that out into 4-hour increments with weeks of turnaround time each is frankly ridiculous.

Giving a real life problem of the business is one of the best ways to determine if someone will be good at, shockingly, solving real life problems of the business. I do this frequently in interviews (although I ask for general design principles and not 4 hours worth of code), and every single time it has been a problem I have already solved. If I hadn't already solved it, the test would lose almost all probative value.

Whether a 4-hour test is worth it to you is a personal decision. I was always taught to treat looking for full-time work as a full time job. When you're unemployed or underemployed and spending 8 hours a day looking for work, a 4-hour coding test is nothing. I've spent far longer than that on tasks like brushing up on rusty languages, writing portfolio programs, and customizing resumes for specific positions.

On the other hand, some of the best workers are already gainfully employed, and only casually looking for better opportunities. People in that situation are unlikely to go through the rigamarole of a 4-hour test, unless the opportunity is stellar. However, that's the company's problem, not yours.

As far as discerning what it means about the company's attitude toward their employees, I don't think you can really say anything either way, other than they are probably tired of dealing with unqualified applicants, to a degree that they're willing to throw out some of the good with the bad.

Answer 2

No, not typical, why are you solving their problems for free? (4 hours)

1 hour is typical for a programming test. In the past our programming test was 4 questions. First 3 questions took 1/2 hour, last one 1/2 hour. We also gave the test to existing hires in house to make sure we were in the expected timeframe and the test was fair and adjusted accordlingly.

The first few question(s) were "Fizz Buzz" type to weed out people who can't program. They got progressively harder. The last question was a problem solving exercise. Generally we tried to limit the amount of code written to around a few hundred lines (total) and not require any clever tricks. We also scored people on error handling, style, syntax, organization, etc. The questions were not related to our business but rather the skills and technology that the current platform was written in.

Typically, great candidates finished in less time than was allotted. Sometimes people requested extra time which we allowed because of the stress involved in taking a quiz, but we capped everyone at a certain limit. The quiz was in the current development environment and people had access to the internet for reference information. We also went over the expectations of the quiz to every candidate.

At one time we did discuss incorporating our code base (real world) into the quiz but we eventually discarded that due to concerns that code copied off/stolen/etc (our boss was a bit paranoid). Eventually we just went with a separate quiz.sln in an isolated development machine.

Finally, we found it was hard to come up with a test that was fair, but neither too hard nor too easy. We always asked our candidates about the quiz after they took it and garnered their feedback to refine it for future candidates.

Answer 3

I find coding tests in interview are a load of tosh anyway. No-one codes anything but the simplest routine under pressure without the usual environment and tools, so the results you get are dubious at best.

What I have found to be really good tests of a programmer's ability is to give him some project code and ask him to review it, this works really well if the code has several obvious bugs, several obvious code issues, and a few questionable practices. A good coder will tell you all of them, and will engage with you in discussion of why some code isn't 'wrong' but could be done better to ease maintenance or so. A poor programmer will find a bug and stop.

Any job that expects you to do a test that takes more than half an hour just hasn't spent even that long working out a good, targetted test that provides them with more than a vague idea of your skills. (most companies find it very hard to spend any time working on pre-interview setup).

If I was given a test like you'd got, I'd write the answer in pseudo-code. That should be enough to demonstrate my understanding of coding and design, without going through the entire compile, build and test phases you would for a normal work project.

Answer 4

You may not have 4 hours, but somebody more interested in their company certainly will. I was essentially hired based on a similar task that a company asked me to do beforehand on the task alone. Apparently, writing clean and understandable code, thorough test cases and understandable and coherent design documentation is an abnormality. Actually seeing someone do it blew people away. Anyways, everyone I spoke with at the interview commended me on what I did and I felt like I had to impress nobody in the interview because they had already made up their mind. It was simply a matter of me not giving them a reason to say no by doing something stupid.

So while I agree 4 hours is a fairly large time investment, it also means the task is of sufficient size that you have an opportunity to really show what you are capable of. Your work very well may speak volumes more than you ever could in an actual interview situation.

As a side note: I have tried a similar thing lately but using a much smaller problem and I haven't been happy with the results. Small problems are to trivial to demonstrate enough of the person's knowledge. Plus, trivial problems tend to require the person to recognize some trick/detail necessary to solve the problem. Thus, there is some balance between taking up too much of a person's time versus not gaining any real benefits because the task is to trivial. I would think a 4 hour task is probably the right amount of time to be complex enough for candidates to demonstrate their skills and not be so long that nobody would bother.

Answer 5

I took a 6 hour coding test at one point. When I took this test I had fairly high confidence I would be hired - while it came true, I wasn't all that satisfied with the follow-on.

Obviously having lots of employers each asking for 4 hours is excessive. What the person was looking for in the test I took was my coding style - I was hired because mine was 'closest' to his. In this context, look at the problem from this perspective: First, is it an interesting problem that solving is worthwhile to you in any case? After all, you could learn something valuable.

Second, if you can 'pass' the test does it mean you're hired? If this isn't fairly obvious then you have to decide whether there are other reasons to do it anyway.

Third, they might estimate that it takes '4 hours', but you might find out differently. Do they really know how long this should take? Most likely the answer is no. Therefore, they are going to keep testing people on 4 hour deadlines until they realize it won't fit in four hours. In that case you're wasting your time. The best approach then is to get aggressive with the hiring manager, and figure out whether you should stop at four hours and give them what you have, or continue until it's done and tell them how long it took. In short, there may be a character test wrapped up in this, and simply trying to accept it on their terms may reveal inexperience.