It's part of my job to interview new candidates and I came up with a test that pretty much measure the coding skills of the candidates. However I couldn't (yet) come up with a good question to measure the candidate's capacity to deal with abstraction.

Earlier I had the following question in my test:

Suppose a tree structure where each node stores an integer value. Draw the simplest Class Diagram using UML that represents the domain model described.

Then I'd ask:

Now change the model on question above to represent a leaf (i.e. a node that has no children).

Eventually, after several interviews, I realized those two questions were not giving me any clue if that candidate knew abstraction. Some people knew the answer but during the interview showed me they actually don't have a clue when it comes to abstract more complex subjects.

I can't really have a very deep complex question in this test because:

  1. The total time for the entire test is ~2h and they already spend about 1h to 1h30 in the first part (coding skills)
  2. A good candidate might fail in a specific complex question and that would not really prove they can't abstract at all

After reading this article I got intrigued when he says:

Inventing questions that force candidates to understand pointers without using C isn’t too hard. Nearly any question that forces candidates to invent a data structure (e.g., a hashtable, an AVL tree, or the like) will test how they handle indirection, the idea that having a thing is different from having a pointer to that thing. So I’ve picked a question that forces candidates to design a data structure. And, sure enough, I see candidates who have a lot of programming experience, but who don’t “get it”, completely bomb out in my interview.

The way I see it, inventing a data structure is a good way to measure abstraction skills.

So my question is, does anyone know a good question (or a set of small questions) that could measure for abstraction skills in a test?

I'm looking for those kind of questions that:

  1. Don't depend on any language in particular
  2. Can be answered by smart people
  3. Can't be answered by people who know all books by heart
  4. Will take average 40 min to solve
  5. Will not produce huge amount of pages as an answer

closed as primarily opinion-based by gnat, durron597, user40980, user22815, Michael Kohne Jun 11 '15 at 12:33

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Abstraction is too broad a topic. Arguably, covering the entire core curriculum of a CS degree. The range of complexity varies as much as life itself. Even children are taught about Abstraction. Look at the animal kingdom classification. You need to ask a more open ended question to avoid turning this into trivia. – P.Brian.Mackey Apr 15 '11 at 23:20
  • Let's test an assumption here: why is "ability to abstract" a more important indicator of technical competency and suitability that a real problem from your actual domain? – Rein Henrichs Apr 16 '11 at 0:14
  • At one point Alex talks about pointers. While "ability to abstract" might be unnecessary for some grunt-type jobs, understanding pointers/references is presumably pretty essential. I agree that a real problem might be good to avoid the memorised-answer problem. – Robin Green Apr 16 '11 at 7:53
  • @P.Brian.Mackey, exactly, abstraction is thought since you're a child hence my difficulty in measuring it. Anyone can abstract, not necessarily developers. In my opinion a good developer will have solid technical skills and will usually have very strong abstraction skills. The kind of person that when you explain how the solar system works they can abstract the concept and instantly understand you have thousands of solar systems interacting in one big universe. Some people don't find so easy to grasp such concepts. – Alex Apr 18 '11 at 21:31

At the risk of exposing one of my treasured interview questions, I've always fallen back on interesting experiences directly taken from previous jobs. For example, one question that I've found useful is to ask candidates about how to create a file system that is resistant to file system corruption due to power loss. That's because, for a while, that's what I was working on, so I'm able to judge approaches and ideas and provide additional information. And they'll need it, because for the most part, I present the problem pretty much as I describe it here. This question has a few desirable attributes with respect to interviewing:

  • It's language neutral. I'm not asking them to code for a particular environment, I'm asking about the ideas behind the problem.
  • It's outside of most folks' comfort area. This forces them to think about how a file system works. Why would a file system have corruption problems with sudden power loss? What can be done to solve the problem? A candidate that isn't able to ask me good questions and obtain more information is probably not the right candidate for me.
  • Finding a reasonable solution for most folks will involve reasoning by analogy. Most people who come to a reasonable solution here think in terms of databases, and reach an approach that involves transactions, the same way that they'd combat data loss in a database. That's a good approach, and someone thinking along those lines displays a flexible method of thinking about computing problems. There have been other successful lines for people to move along, but again, it usually requires people to demonstrate taking successful techniques from other domains and cross-applying them. I want those kind of people!
  • It's easily extensible. You can go pretty far down the rabbit hole here, expanding the question to take up a lot of additional time. You can even lead them into coding questions from here (have people write a file system cache in an interview, it's fun!) without stretching your imagination much.
  • For me, because I've done this professionally, it's easy to judge the results.

I've never been afraid to throw a real-world problem at people in an interview. Sometimes folks are going to be completely flummoxed by it and will have trouble getting started. For the most part, that's not someone I want on my team. A good candidate will take the bare threads you're giving them, ask questions, reason their way along and assemble an approach. It's usually pretty easy to get a lot of information about people's abstraction skills using a question like this.

  • real problems also help concrete thinkers who can code but don't deal well with extremely abstract ideas. I know I often struggle with questions based entirely on hypotheticals. "if I wanted to solve a problem..." type questions often leave me questioning the sanity of the interviewer. The "design a ..." concrete question is a better approach if you want people who actually solve problems, rather than suffering analysis paralysis. – Мסž Apr 16 '11 at 9:31
  • Thank you @jjb, I think you nailed an excellent approach to capture the essence of abstraction using a technical problem. I will have to come up with a similar question and test it on real candidates to get any kind of measure but it seems to me that this is a pretty good approach. Thanks for sharing! – Alex Apr 18 '11 at 21:28

Not the answer you're looking for? Browse other questions tagged or ask your own question.