What's a good question to measure the candidate's capacity to abstract? [closed]

Question

It's part of my job to interview new candidates and I came up with a test that pretty much measure the coding skills of the candidates. However I couldn't (yet) come up with a good question to measure the candidate's capacity to deal with abstraction.

Earlier I had the following question in my test:

Suppose a tree structure where each node stores an integer value. Draw the simplest Class Diagram using UML that represents the domain model described.

Then I'd ask:

Now change the model on question above to represent a leaf (i.e. a node that has no children).

Eventually, after several interviews, I realized those two questions were not giving me any clue if that candidate knew abstraction. Some people knew the answer but during the interview showed me they actually don't have a clue when it comes to abstract more complex subjects.

I can't really have a very deep complex question in this test because:

1. The total time for the entire test is ~2h and they already spend about 1h to 1h30 in the first part (coding skills)
2. A good candidate might fail in a specific complex question and that would not really prove they can't abstract at all

After reading this article I got intrigued when he says:

Inventing questions that force candidates to understand pointers without using C isn’t too hard. Nearly any question that forces candidates to invent a data structure (e.g., a hashtable, an AVL tree, or the like) will test how they handle indirection, the idea that having a thing is different from having a pointer to that thing. So I’ve picked a question that forces candidates to design a data structure. And, sure enough, I see candidates who have a lot of programming experience, but who don’t “get it”, completely bomb out in my interview.

The way I see it, inventing a data structure is a good way to measure abstraction skills.

So my question is, does anyone know a good question (or a set of small questions) that could measure for abstraction skills in a test?

I'm looking for those kind of questions that:

1. Don't depend on any language in particular
2. Can be answered by smart people
3. Can't be answered by people who know all books by heart
4. Will take average 40 min to solve
5. Will not produce huge amount of pages as an answer

Answer 1

At the risk of exposing one of my treasured interview questions, I've always fallen back on interesting experiences directly taken from previous jobs. For example, one question that I've found useful is to ask candidates about how to create a file system that is resistant to file system corruption due to power loss. That's because, for a while, that's what I was working on, so I'm able to judge approaches and ideas and provide additional information. And they'll need it, because for the most part, I present the problem pretty much as I describe it here. This question has a few desirable attributes with respect to interviewing:

• It's language neutral. I'm not asking them to code for a particular environment, I'm asking about the ideas behind the problem.
• It's outside of most folks' comfort area. This forces them to think about how a file system works. Why would a file system have corruption problems with sudden power loss? What can be done to solve the problem? A candidate that isn't able to ask me good questions and obtain more information is probably not the right candidate for me.
• Finding a reasonable solution for most folks will involve reasoning by analogy. Most people who come to a reasonable solution here think in terms of databases, and reach an approach that involves transactions, the same way that they'd combat data loss in a database. That's a good approach, and someone thinking along those lines displays a flexible method of thinking about computing problems. There have been other successful lines for people to move along, but again, it usually requires people to demonstrate taking successful techniques from other domains and cross-applying them. I want those kind of people!
• It's easily extensible. You can go pretty far down the rabbit hole here, expanding the question to take up a lot of additional time. You can even lead them into coding questions from here (have people write a file system cache in an interview, it's fun!) without stretching your imagination much.
• For me, because I've done this professionally, it's easy to judge the results.

I've never been afraid to throw a real-world problem at people in an interview. Sometimes folks are going to be completely flummoxed by it and will have trouble getting started. For the most part, that's not someone I want on my team. A good candidate will take the bare threads you're giving them, ask questions, reason their way along and assemble an approach. It's usually pretty easy to get a lot of information about people's abstraction skills using a question like this.