Is over-engineering a warning sign? [closed]

Question

So we present a straightforward coding exercise to new candidates with some well defined requirements. Occasionally we receive solutions which don't really solve the problem at hand, but are over-engineered to solve a perceived problem - often outside the bounds of the exercise.

Now my question is, is this a warning sign?

EDIT: Quite a lot of the discussion is based on the test being flawed - which is a fair point. As I described in a comment, the basic premise of the test is to show how you can read the data from the file in a sensible way (and you'd be amazed at the variety of approaches we see), and how to match the items before calculating the latency between the updates. Now for this to work, certain assumptions have to be made about the data, and we look for these assumptions, and we also state explicitly that we want to see the approach you take (including OO approach etc.) All this in a two hour time frame.

IMHO, when I was interviewing it was the most complete exercise I came across.

The particular scenario which I'm pondering about is where a candidate, rather than reading from the file, accepted "network" input in a multi-threaded application, which clearly is not in scope.

Answer 1

The problem is the test is skewed. You're asking someone to demonstrate their ability to write complex, enterprise-level software using a simple exercise taking only a few minutes. There are other interviewers at other companies who complain that candidates don't show enough skill in object oriented design with these exercises, so people tend to overcompensate. It doesn't necessarily mean your candidate is incapable of using simpler code when the situation warrants it.

If you want to know if that's the case with your candidate, just ask them to redo it, giving them some specific guidelines. Say, "I can see you were showcasing your object oriented design skills, but it seems overkill for such a simple problem. Can you rewrite it using only two small functions?"

Answer 2

I would say that this is a clear warning sign, but not necessarily disqualifying for a candidate.

There are two separate problems that candidates seem to be having:

1. Missing the point of the exercise - This is fairly alarming. All of the programming skill in the world won't create a good result if someone is unable to cast the problem that they are solving properly. I have found that the most productive engineers are the ones that are able to identify the true problem to be solved, even if it isn't exactly the problem posed. Having the ability to think critically about the question being asked, and perhaps reformulate it into something that is simpler to solve is a critical skill. Missing the point when the problem is being clearly stated should be a big red flag.
2. Overengineering the solution - This is a secondary issue and is often the result of the first issue. There a couple of different pathologies that can have this result. First, not properly understanding the problem, or casting it too broadly can result in a solution that is just too complex. This is clearly related to the first point above. Second, trying to "show off" by thinking through future scenarios that might not really be relevant. This can be problematic because it indicates that the candidate hasn't understood the value of simplicity in solutions and deferring complexity until it is truly necessary. This is something that can be corrected with good guidance, but is a watch out when bringing someone into the organization.

I would suggest following up with the candidate about their answer during the course of the process. Rather than simply looking at their result, try to ascertain what led them to the result, and given some guidance, how they would respond and change their answer. This will probably be more revealing about their capabilities than just their direct response to the problem. The "whys" of their approach can give you a sense of whether they are just not getting it, or if their understanding was just slightly off in how they chose to respond. This kind of follow up will also reveal more about their overall approach to problem solving.

Also, reexamine the problem itself and see if maybe it is unclear and perhaps causing folks to get on the wrong track as they formulate their answers.

Answer 3

No, not during a job interview. Too many interviewers do things like intentionally underspecify the requirements and expect the interviewee to ask further questions, or look at attention to real-world issues not explicitly mentioned.

Here are some things, off the top of my head, that your requirements probably didn't mention:

• Coding standards

• Comments

• Exception handling

• Descriptive names of variables, classes, methods

• Following idiomatic usage of the language

• Proper object oriented design

• Attention to possible concurrency issues

• Writing test cases for the code

Did you pay attention to a single one of these things without explicitly stating it? How would the candidate know which ones you care about, and which you don't?

An interview is a highly artificial environment. The interviewer is often trying to "trick" the candidate a little bit in order to make it difficult for the interviewee to tell him whatever he wants to hear, and the interviewee is trying to guess what the interviewer really wants.

Generally, making a mistake on that guess is pretty different than making a mistake on real world design decisions. If you want to know if someone over-engineers things you probably have to talk about design rather than look at a very artificial coding exercise.

Answer 4

IMHO the answer is clearly yes, it is a warning sign, if

• the coding exercise had a clear task
• has well defined (ans also well-written) requirements,
• the candidates had a chance to ask questions to make sure they solve the correct problem.
• you want people who are smart and get things done in your team, not architecture astronauts.

Answer 5

Not nearly as big of a warning sign as not solving the problem at hand. The fact that he failed the quiz and didn't provide the correct solution is a warning sign. It's not necessarily a no-go scenario because it depends on how and why he didn't provide the correct solution.

A lot of times the question is complete crap and simply unanswerable. Don't pretend that interviewers never make mistakes. It's still a failure on his part to suss out why the question is crap. If you're hiring an engineer who is expected to help make requirements, this is a problem. If you're hiring a coder, you just don't expect him to do that.

Presuming that the coding exercise isn't horribly messed up, it appears that the way he failed it was misinterpreting the problem and going off into the weeds trying to solve a problem that wasn't there. Yes, that's a warning sign.

Answer 6

Maybe.

Not solving the problem is of course a clear cut warning sign that something is wrong. What went wrong? Either they misunderstood the problem, or they made a bad solution. A bad solution for something simple is a clear sign that the candidate is poor.

If they misunderstood the problem, then take a long look at your requirements. Even things that seem crystal clear to you may be unclear to others unfamiliar with a domain, or from a different background (locale, age, upbringing). If the candidate was there in the room with you, or offered an opportunity to ask questions and still failed, I would consider that a failure on their part. If the requirements were thrown over the wall, I would likely give them the benefit of the doubt (and think about fixing the interview process).

If the solution was correct, then it becomes less clear. Personally, I think that a number of people take YAGNI too far. If you can take a specific problem and create a general solution without increasing complexity or harming maintainability then why not make the general solution? (Think reversing a string vs. reversing any collection) That sort of "over-engineering" is clearly good in my opinion.

Everything else is that grey middle ground. Does the solution address likely axes of change? Does the solution add complexity or harm maintainability? Adding a little complexity to solve future problems that are nearly guaranteed to a win. Harming maintainability greatly to account for something that is totally unlikely is not.

Being a good software engineer means striking the right balance here. Being a good interviewer means striking the right judgement/inferrence about how and why the candidate chose that balance, often using other parts of the interview to evaluate.

Answer 7

Maybe but consider the following:

• Interviews are hard: People are under stress. This should be factored heavily into any problem you give someone

• Requirement Length: How long and straight forward are the requirements? Did you make them extra wordy to make sure you included everything. As a result, they may be clear to you but the requirements maybe over engineered! I took a job interview once for an hour problem with about 3 pages of text for a problem that was relatively simple. Sometimes all that text can be tough to read and interpret on an interview and can be misinterpreted as well.

• Sometimes Less is More: I would much rather have a few bullet points, sentences, and or examples showing the main requirements and then a discussion with someone to ask questions to and maybe reach out to along the way if needed. I guess what I am trying to say is you should first check your test requirements are not overly complicated for a simple problem.

• Communication Skill: you should be testing the candidates ability to communicate and ask intelligent questions about the problem first and once you know they have demonstrated that they understand the problem then you can set them forth on their implementation.

• Bottom Line: If you haven't checked that the problem is understood than you really don't know what to make of the over engineering. As others have said it could be a good or bad thing but if you didn't check the understanding or communicate with the candidate about the problem up front, it is harder to know their general understanding of the problem and what to make of it.

Answer 8

A lot of this could be attributed to how you word the question and how you've put the entire interview into perspective.

It's like, "Let's see how creative you are. What is 2 + 2?" Four? All you could come up with is the most simple, obvious and accurate answer? Young / entry level developers who are so eager to impress during an interview will take the "We want to test your coding skills or see how good of a programmer you are." to mean "do something very sophisticated." We all like to think simple is best except when it makes things more difficult.

There are ways to see if someone is prone to always be over-engineering things. Give a code example of something that was too complex and ask for a simpler solution. When someone provides a solution that you don't think will work, this is a great opportunity to see how they react to criticism. Personally, I'd like to see someone be open to new ideas and recongnize a better solution than someone who is going to make the same mistakes over and over again.

And in reality, don't we always have an opportunity to change our code when it doesn't work? You can under design or over design initially. Once you've recognized the simple solution, shouldn't it be easier to implement?

Answer 9

It depends, but generally no.

Design in general is a skill learned with experience. Aaronaught's description of that progression linked by Marjan is generally a good one.

Communication in any form also depends a lot on context. What may seem perfectly clear to mean one thing in one context, may just as clearly mean something else within a different context. Knowing which questions to ask is also something that comes with experience.

Their thought process and how they reason about their decisions is far more important than their solution. Without reviewing their solution and their decisions with them you cannot fully assess the context it was developed under.

Given the progression above, a candidate with the over-engineered solution may well be further along than the candidate with the simple one.

Also: What level position are you hiring for? Over-engineering out of an entry or intermediate level candidate is less of an issue than over-engineering out of a senior level candidate.

Answer 10

If the programmer didn't solve the problem, then all of the extra code is the coder's attempt to obfuscate the non-answer. It's the same technique used on an essay test by a student who doesn't know much about the subject. He or she will ramble on about a compelling, but unrelated issue in hopes his ignorance is masked by word count.

If the programmer did solve the issue but added a great deal more code, consider the programmer's background, as some areas of programming require greater tolerances than others.

For example, code running the auto-pilot in a commercial passenger jet has much less tolerances for failure than a free Android game. A developer used to programming embedded devices that are hard to reach and almost impossible to update will tend to code more what-if's then a developer that can push out 15 updates in a day.