154
votes

I am looking for a job and have applied to a number of positions. One employer responded. I had a pretty lengthy phone interview (perhaps an hour +) and they then set me up with a developer test. I was told that the test was estimated to take between 6 and 8 hours and that, provided the results met with their approval, I would be paid for my work.

That gave me some pause, but I endeavored. The developer test took place on a VM accessed via RDP. The task was to implement a search page in a web project that requests data from the server, displays it on the screen in a table, has a pretty complicated search filtering scheme (there are about 15 statuses and when sending the search to the server you can search by these statuses) in addition to the string/field search. Additionally, they want SVG icons to change color on certain data values, and some data represented differently than how it's structured in the database.

Loooong story short, this took a heck of a lot longer than 6-8 hours. Much of it was due to the very poor VM that I was running on (Visual Studio 2013 took 10 minutes to load, and another 15 minutes to open the 3 GB ginormous solution).

I was told that after completing the test I should commit my changes to source control... Hmm, OK. I followed directions. And after committing the changes, I was emailed a response. The SVGs weren't colored right, there was a bug in this edge-case, there was an occasional problem with this other thing that I never experienced, etc. So I am 13-14 hours into this thing now, and I have to do bug fixes. I do them, and the employer comes back with more bug fix requests.

All my work is apparently going into a production application. I noticed a few anomalies in the code where it looked like others had coded all of one functionality but hadn't touched anything else.

Am I just being used for cheap labor? Even if they pay me the promised 50 dollars an hour for 6 hours, I have committed about 18 hours to this thing now. If I bug fix all of the stuff they keep coming up with, I will have worked at least 16 hours for free.

I have taken a number of developer tests, but I have never taken one during which I worked on code destined for production. I have never taken a test where I implemented a feature that was in the pipeline for development, and I have never taken one that took 4 rounds and a total of 20+ hours. I get the impression they are using their developer test to field some of the functionality on the cheap.

Do I have the wrong impression? And is this testing protocol appropriate?

38
  • 165
    That's crazy. I think you were probably taken advantage of for free labor.
    – jmq
    Jun 11, 2014 at 2:03
  • 79
    You should send an invoice. Jun 11, 2014 at 9:33
  • 27
    The good old interview-driven development. There should be a "devabuse" tag.
    – Den
    Jun 11, 2014 at 11:23
  • 74
    Here's a suggestion, for people in the US. Whenever a firm asks you to submit code samples, or do example work for them, every chunk you submit should carry a copyright notice: "COPYRIGHT (year) (your name). ALL RIGHTS RESERVED." If it generates code that will be run in a user's browser, make sure that the generator also generates a comment line containing that copyright notice. Honest outfits will have no problem with this. Dishonest ones will scream. Jun 11, 2014 at 12:59
  • 32
    Consider reporting this company to your state department of labor. Taking work for free is generally illegal.
    – nobody
    Jun 11, 2014 at 14:49

11 Answers 11

168
votes

I would never participate in a code test of this nature. I have taken many code tests and done many code projects. I certainly wouldn't check code into someone else's repository under any circumstance. If they don't know what they need to know after a 4 hour sample with some minor bug correction in a pair-programming session, then they won't ever know.

Going into a test, you should know and make clear a few things up front:

  1. It should be agreed upon and known that any work produced during the test may not be used for any purpose other than determining your skill at the required tasks.
  2. A code test should not last more than 4 hours.
  3. You are not an employee of the company, so any suggestion that you might be paid for code produced is preposterous. Insist on a written contract of payment if there is even a hint of this.
  4. Set specific limits on the time you will spend on any given part of the test, and then stick to those limits. If you find yourself going over the limits for any reason, consider why you are going over that limit. Is it because of pressure from them? Is it because you've made mistakes? Is it because you've poorly estimated how long something should take to complete?
  5. Stand your ground if you feel you have covered a particular topic. If you've already fixed a bug, and they're asking you to fix a nearly identical bug, say "We've already covered that topic with bug x, perhaps we could move to something else that demonstrates something new."
  6. Under no circumstances should you check anything into a production pipeline. This includes into any kind of development branch that may ultimately lead to a production pipeline. When in doubt, check nothing in. For code tests that are not necessarily in person, I insist that the code be checked into my personal public repository first. This gives me at least some kind of protection from having my work used inappropriately.
  7. Judge them for their behavior every bit as much as they are judging you. If you feel they are not being up front with you, call them on it. If you feel you are being mistreated, speak up.

The company you are interviewing with is also being interviewed by you. If this is how they are treating someone they are interviewing, is this a company you want to work for? I understand that often people have a need for a job and often this need will override some common sense concepts, but this should always be in the forefront of your mind. Don't be afraid to walk out. If it doesn't feel right, follow your instincts and vote with your feet.

12
  • 10
    I have already determined that I am not going to work for them. Having see the monstrosity of a solution that they are using (and the amazing number of layers of abstraction ON TOP of Entity Framework) I have no wish to kill a little part of me every day by doing the things they are doing. I'm having a phone meeting tomorrow where we are going to discuss the problems that I have had with this "test". Jun 11, 2014 at 3:46
  • 8
    worse, if I were a company I'd not want people not working for me and having signed all the proper NDAs and other forms to have access to my production environment, and your VCS is a production environment...
    – jwenting
    Jun 11, 2014 at 5:08
  • 18
    In the UK at least, I have a strong suspicion that making an interview candidate do real work is either illegal or legally actionable by the candidate. Either way, it is considered very bad practice here, and we spend much time creating "fake work" tests that are thorough enough to evaluate a candidate, but have no other practical use. Jun 11, 2014 at 8:16
  • 9
    @marcvangend: My primary concern here (US based) would be liability. Since there is no formal contract and no direct relationship, if something goes wrong and something in their software is exploited, it would be very simple for them to place blame on some "interviewee". #1 is a self protection measure only rather than an attempt to prevent my work being stolen. I've had plenty of my work stolen over the years, and it really doesn't bother me. Jun 11, 2014 at 10:56
  • 10
    @WayneWerner: Or the traditional consultant's rhyme. "Roses are red, violets are blue, pay my ass." Jun 11, 2014 at 21:09
46
votes

Many interviews are followed by tests. Those tests are needed to ensure that you actually have the required skills and to give a better view of a few things which are difficult to test during the interview itself (such as do you apply style rules to your code).

This being said, a test is a test.

  • It doesn't need to be long. There is not much you can see after eight hours of coding that you cannot see after thirty minutes. More importantly, code written during the test should then be reviewed, line per line, which takes an important amount of time. It is not unusual to spend more than two hours to review the test code written during half an hour.

  • It shouldn't deal with an existent code base. Understanding the code base of a medium-scale product may take days or weeks (or months or years depending on the code quality and technical debt). Intellectual property may also be an issue (unless the code is open sourced).

    When the goal is to test how the candidate is able to maintain the existent code base, the test can be done on a small (500-600 LOC) fictional code base written specifically for the tests.

  • It doesn't have to be a request to develop a real-life app or feature. It can be a completely useless piece of code, written with the sole intention to show that you've understood the problem and found an elegant way to solve it.

  • It doesn't have to be perfect. There are bugs? That's fine. Take a note of them for a further interview with the candidate; it can be an excellent opportunity to see how the candidate reacts in this situation.

  • It doesn't have to be done through RDC on a VM, unless you don't have Visual Studio yourself. If the goal is to see your coding and problem solving skills, it doesn't matter where do you do the exercise.

  • It's out of question for the code written during this test to end up in the version control of the company. Why would they pollute their version control with something written by a candidate?

To conclude, when you're asked to spend dozens of hours writing production code, solving bugs, and committing your work to the version control of the company:

  • Either they are just using you to implement features for free,

  • Or they really don't understand how to do an interview.

In both cases, look for a better place to work.

4
  • 4
    +1 for "or months or years depending on the code quality and technical debt". Plus if it's that bad they will drive away the developers who are capable of sorting out a steaming pile of WTF and have to hire the ones who will make it worse... Jun 11, 2014 at 10:02
  • It's completely legitimate to have a problem like, "Given these EF classes, write a query that returns some complicated criteria and display the results in a form". I can't imagine asking the interviewee to create that kind of environment themselves rather than just give them an existing VM with everything all set up.
    – Gabe
    Jun 13, 2014 at 18:24
  • @Gabe: good example. But in this example, it's also obvious for the candidates that you're not looking for a cheap workforce to implement a feature in your new product, and that the criteria is not so complicated that it would take eight hours to obtain a solution. Jun 13, 2014 at 21:34
  • You're completely right; I'm just saying that under any circumstances the interviewer should expect to provide whatever environment that is needed to complete the given task.
    – Gabe
    Jun 14, 2014 at 4:34
22
votes

Not going to write a long answer, but I am seriously confused, why is nobody bringing up the issue of copyright?

As far as my experience goes I have never heard about an agreement being made to transfer copyright ownership of code written during a developer test to the other party. If this is the case you can actually sue them for copyright infringement and the damages awarded for this can be quite nice, especially so in the US from the stories I have heard. And if they want to settle (do propose this) you can ask any exorbitant fee for the infringement (after which they would in principle still not be allowed to use your work and you could still sell your work to them if they would still be interested).

10
  • 14
    I've never heard of a coding test making its way into production. Copyright is a valid issue here.
    – Josh
    Jun 12, 2014 at 6:39
  • 1
    Not to mention insurance
    – Crake
    Jun 12, 2014 at 16:01
  • 1
    Copyright is an interesting issue, but perhaps only if the work turns out not to be paid and it is actually used. If payment is eventually given, the copyright is probably meaningless. But storing a copy somewhere other than the interviewer's internal repository is called for. Ideally the personal copy should also be timestamped before submitting to interviewer's repository. Various suitable internet repositories could be used while keeping the code from being public. Jun 17, 2014 at 4:57
  • @user2338816: First of all, even if payment is given you can still sue for copyright infringement as the payment on paper is often not a work contract, but rather a... ok, can't find the english term. You're right though that you will need a copy of the code as you first need to register it with the copyright office before you can file for a lawsuit. Jun 17, 2014 at 12:57
  • 1
    A cautious +1 from me. Being overly aggressive seems unethical to me - you are entitled to your rights, but I don't want to be the kind of person that sues to burn the interviewing company down, and I wouldn't hire someone who had such a reputation, either.
    – Kevin
    Jun 18, 2014 at 21:27
12
votes

People with more career experience may be better able to answer this question, but I personally would not be very comfortable with a 20+ hour dev test. It sounds like they are using the interview to get work tasks completed.

I am assuming that you have not signed any legal documents concerning ownership of the code. So I would wait until they reviewed the code and accepted or denied it. Then if they accepted it I would ask to get paid for the full time, 20+ hours. I'm not sure that I would take payment for just the six hours that was originally suggested. If this is going to go into production, then they will need to straighten out code ownership.

At the very least, discussing payment for the code should help you decide whether you would want to accept an offer. I wouldn't want to accept an offer if they thought only paying you for six hours was fair.

2
  • 3
    I have determined that I am not working for this company. Even outside the test, there are some real problems with the way they are doing things. The project I was working in was a solution with 132 projects in it. I'm told that a dev on the team made the case for it to be structured that way, so they did this on purpose! They want all work done over RDP. I live in Hawaii and the company is East Coast, that is not going to work. Then the test seems more like I am a temp hired on the cheap. Jun 11, 2014 at 4:03
  • One of my former coworkers went on to be part of a team with a solution with 200+ projects in it, so this is not unheard of.
    – trysis
    Sep 19, 2014 at 21:53
11
votes

When I was in a position to interview developers, those tests were short and simply "pass or fail", no bugfixing included, even when there were a few minor bugs in the code. That's because I wanted to assess the candidate's skills, not get a production ready piece of software.

The situation described in the question looks very much like someone is trying to get something useful for free (or cheap).

7
votes

I've never done a dev test more than an hour long, and these have all been 'puzzles', a piece of work to see if I can solve problems and meet a stated aim within a given time limit.

$50 (or to me, £25-30) is a pretty poor day rate, it's like asking a plumber to fix your toilet in exchange for a drink.

My advice, in no uncertain terms, is to blog about your experience, referring to the company by name in case they're trying to create a whole app with this technique (people often google companies before heading to interview) and don't let it happen again. Next time they ask for a bug fix, you name a consultancy day-rate (at least 5 times what they've been offering) and make it known that developers will not work for free.

Being fooled is, sadly, part of life, but you don't have to sit back and accept it.

9
  • 5
    bad advise to tell people to name&shame like that, can end someone in no end of legal and career trouble.
    – jwenting
    Jun 11, 2014 at 9:58
  • 1
    I actually just got off the phone with them. Having turned the position down I was asked if I could provide my criticisms, and I explained that I was worried about the possibility that I was developing on the cheap. Based on the feedback that I got from the POC at the company, and checkins in their source control and further looking in the source code, I don't think they are doing this much. Looks like 2 people per 6 months or so, about on average for a small company's hiring trends. That said, I still don't agree with the practice. Not gunna work there. Jun 11, 2014 at 16:03
  • 8
    @CodeWarrior I really think that you should threaten them with legal action. Unless you signed a contract handing over the code you have copyright of what was written during the 'test' adn they are effectively stealing it from you. Companies like this will continue in their nasty practices until someone calls them on it. If you don't step up then what about the next poor sap they 'interview'? Jun 12, 2014 at 7:37
  • @CodeWarrior You say you turned the position down. Does that mean they offered the position to you? And has there been further discussion of payment for your "dev test" work? Jun 17, 2014 at 5:02
  • 1
    @CodeWarrior Good call! Sometimes you just need to know when to walk away.
    – AJFaraday
    Jul 11, 2014 at 9:05
3
votes

Just for comparison: The interview for my current job was around 1 hour talking about what I did so far and what the company is planning to do and how I would fit in. After that we worked together a week on a small project they had lying around, I guess just to see how we get along with each other. They paid me for this as freelancer the same amount I get now as their employee, so there was never a full day of unpaid work, let alone 3 days.

If the code is really used in production, I would send them the bill for the 24 hours you have spent, not your fault if their estimations are wrong. Assuming they didn't let you estimate how long it will take.

2
  • When you say you worked on a project they "had lying around", was this a project they used in production? As other answers indicate, this is strange in any situation, whether you're paid or not, and usually worse for the company than the candidate.
    – trysis
    Sep 19, 2014 at 22:12
  • Sounds like a short version of contract-to-hire, which is a normal business practice (although not usually only one week).
    – Joe
    Sep 22, 2014 at 4:34
3
votes

While you are supposedly getting paid for (some of) your work, this does not sound like a trial project, it sounds like a scam to get cheap/free work out of you. It may be that it is was intended to be a trial project, just not structured or managed very well.

But managment that is so bad that it is sounds like a scam, is definitely something you should take into consideration when deciding whether to take the job or not.

A proper trial project should make it clear that

  • They have work that they desire to have done.
  • Based upon your interview they believe you should be able to do the work.
  • Sucessfull completion of the project does not guarantee a position.
  • Terms for the project (how much they will pay, who owns the code, whether it is time and materials or flat rate, estimated time to completion, etc).
  • The project will be reviewed and feedback provided -- not just a yes/no as to whether or not you get the position.

The terms should be acceptable to you regardless of whether you get hired -- if the terms are only acceptable if they come with a full time job, they aren't really acceptable.

2
votes

I don't think they would actually use this to get cheap labor.

The reason is simple. After you write those tests, they need people to review what you write, yes reviewing code is much easier than writing the code itself, but it still takes lot of time.

And then after that they probably need people to maintain those tests, explain it, etc..

And I simply cannot imagine any IT company that would care about saving less than $100 especially companies in the US. It's never how business runs.

10
  • 13
    If they got 20+ hours work done, and paid for 6-8 at the (not stunningly high) price of $50 ph, they have saved a lot more than $100. Perhaps they are more interested in getting something done cheaply and quickly than producing code that works or is maintainable? Don't underestimate the greed, incompetence and short-sightedness of people out there.
    – jwg
    Jun 11, 2014 at 11:25
  • 9
    I'm not arguing that is is advisable or worthwhile, just that it plausibly might happen. There are a lot of scumbags out there, including some rather foolish ones.
    – jwg
    Jun 11, 2014 at 13:42
  • 3
    @randomA: Thing is, you're assuming a real dev team with a sense of ethics. Being more jaded (or maybe just sociopathic), I can easily picture a small team puppeteering the whole thing -- they'd do the high-level design, decide what to build (or fix), and farm out the grunt work as "developer tests". It'd save $thousands over hiring a bunch of full-time employees. The puppeteers' jobs are secure, cause such a development model makes them indispensable. And if you're already that greedy and myopic, then code reviews and documentation are money wasted. If it looks like it works, ship it. :P
    – cHao
    Jun 12, 2014 at 5:58
  • 1
    Oh, i'm not saying it's a worthwhile model in the long term. But short term, it looks attractive. Management would be on board if you showed them the right numbers (hell, they're probably the ones who thought it up), and legal would want to word stuff very carefully, but if you're actually paying what you say you'll pay, AFAIK you're not technically breaking any laws. As far as the longer-term health of the codebase, you could farm out refactoring as yet another bunch of "tests".
    – cHao
    Jun 12, 2014 at 13:09
  • 1
    @Taemyr: uh, if I understand the OP their codebase is a fragmented mess - something about 132 projects in a single VS solution?
    – thkala
    Jun 12, 2014 at 13:22
2
votes

I'm a great believer in code tests for developers interviewing for a job. However, this sounds like the code test from hell... Code tests should never involve production code. They should be simple and should state that none of the work done will be used by the company.

Clearly, the work you did was on production code. You should be paid for all of your time - at a minimum. Try talking to an attorney and see if he thinks it would be worthwhile to sue them. Many attorneys offer free initial consultations. If fraud was involved, and in this case it looks that way, you would be entitled to quadruple money damages, and you might also be able to get some nice punitive damages on top of that.

By suing them and winning, you will make some headlines and discourage this practice by others in the future - which will be beneficial to all software developers looking for a new position.

1
  • this post is rather hard to read (wall of text). Would you mind editing it into a better shape?
    – gnat
    Jun 18, 2014 at 7:15
0
votes

Coding tests are unfortunately a fact of life. That said, it bothers me to be asked to blow four hours on a coding test as a condition of getting my first phone screening. It's unfair to ask a candidate to invest so much when the company has invested so little in the relationship.

I'm a senior developer, and I can pass their coding test. But I won't waste a bunch of time on it unless the company has shown some personal interest in me. I generally don't complete an application to any company with a big, poorly-written online application form that asks me to re-enter my resume so their poorly-written robot can botch the keyword lookup. I generally don't agree to complete a coding test unless it is brief or they are watching it live and talking to me.

Even if they're not putting your code into production, a company that wants you to spend a whole bunch of time typing before you find out if you are even a good fit is a company that is comfortable taking advantage of you. They are signalling what they want their relationship to be; you are the code monkey. They call the shots. And their interview process is designed to find people who are comfortable with that relationship.

Don't be a code monkey. Walk away.

Not the answer you're looking for? Browse other questions tagged or ask your own question.