Is there a non-subjective way to measure programming skills in order to comply with programming tests?

Question

Before you start reading:

I didn't know if I should post this here or on Workplace, but since my question is more about the "programming language" part I thought it would fit better here. My apologies if it's not the case.

Also, I'm aware there is a similar question here, but the context is slightly different and I thought it would be better to post another question rather than digging out the old one.

---

About a few hours ago, I was asked to complete a "C++ intermediate" test for a job application. Turns out this test doesn't look like it's tailored for "intermediate" programmers to me.

Even though I clearly don't consider myself as a skilled C++ programmer, I don't think I am an absolute beginner either. I've been practicing C++ for a few years now, along with other programming languages and more conceptual things such as algorithms and design patterns. I if were to describe myself as a programmer, I'd say that:

1. I'm not an expert in any language, but so far, I've known enough things to handle every projects that were given to me.
2. If there was something I didn't know, then I learned it "on the fly" and was able to go on with the project I was working on.
3. I have a working experience in C++.

Yet this intermediate test asked me about very specific questions like the impact of the overloading of the operator "new", or string manipulations with strtok, memmove and so on. I know these things can be useful, but even in a business context, I never had to do such things. I had to do other specific things, but too bad, it wasn't part of the questions asked.

(In my context, the application is for JavaScript development with a very strong algorithmic dimension, but since I never used JavaScript, they tested me on C++ instead since I'm going to work more on the algorithm part. So they are not actually testing me on a specific context which would justify the specific questions).

Obviously either my idea of what an intermediate C++ developer is supposed to know is wrong here, or this test isn't made to measure global intermediate skills. I mean, I thought I was an intermediate programmer, but that might be because I took in account my ability to find answers to my questions quickly. So I might be more a "beginner developer with the ability to look like an intermediate developer (given a few minutes)" rather than a true "intermediate developer".

So I'll consider the first hypothesis to be true, and say that this kind of test is quite good at measuring programming skills.

How can you effectively measure your programming skills in a language in order to comply with these tests? What are these tests actually trying to measure and what can I improve to get better at them (and potentially be hired for a job...)?

Answer 1

C++ is gigantic. Not just as a language or a library, but in terms of coding styles and understanding. Under the umbrella of C++, you find OOP styles, functional programming styles, metaprogramming, low-level programming, and plenty of others.

Saying that you're "intermediate" is essentially useless for saying anything about what you really know of C++ as a language. Someone can be an expert on the OOP part of the C++ realm, while being easily be mystified by deep metaprogramming constructs. Someone who is skilled in using C++ like C-with-classes can be confused by the simplest iostream idioms. Some people have used C++ for decades without understanding the specifics of the C++ object model.

There is no objective standard for what it means to be an "intermediate" C++ programmer. It'd be far better to list your actual skills at C++.

---

As for the test itself, it is probably good at finding C++ programmers who are used to a C-with-classes or a low-level style. But I wouldn't use it as a litmus test to decide who is "intermediate" and who is not.

You didn't post the specific questions, but asking about memmove strongly suggests deep, low-level memory stuff. The reason being that in C++, memmove can be exceedingly dangerous, and it can only be used reasonably on objects that fit a specific classification (in C++98/03, that was POD. in C++11, we call them trivially copyable). This is the sort of stuff that people who deal in high-performance memory operations need to know about.

strtok suggests the C-with-classes approach. It is an API that C++ inherited from C. But unlike memmove, there are C++ ways to tokenize strings that are generally superior to it. Using a stringstream and repeated getline calls would be equally as effective, and far safer.

You would only use strtok if C tools were more familiar to you, or if you absolutely need every ounce of performance in your string tokenizer.

---

I have to emphasize mobileink's point in the comments. It is very bizarre for a company to test your C++ knowledge, when they're considering hiring you to write algorithms rather than C++ code.

Answer 2

Completely objective (and just as useful as most tests):

#include <random>
#include <iostream>

int skill() { 
    static std::mt19937_64 gen{std::random_device()()};
    static std::uniform_int_distribution<int> range(1, 10);

    return range(gen);
}

int main(int argc, char **argv) { 
    if (argc != 2) {
        std::cerr << "Usage: skill_level <candidate name>\n";
        return EXIT_FAILURE;
    }

    std::cout << "On a scale of 1 to 10, " << argv[1] << " rates a: " << skill() << "\n";
}

If you want a result that's useful, it's probably going to get somewhat subjective instead. Objective testing mostly tests the ability to regurgitate facts on demand--but what matters for real programming is understanding.

While I think it's fair to state that at least some knowledge of facts is necessary for understanding, I can't imagine anybody thinking it's (even close to) sufficient. As such, you're pretty much stuck with guessing about their level of understanding based on indicators that combine indirection and subjectivity in varying degrees.