Is it ok to use meta-programming even though not all of my colleagues understand it?

Question

I employ a lot of meta-programming to avoid repetitive tasks and build safer-to-use abstractions.

I recently moved to a new job where I am working in a larger team and this worries some of my colleagues, because they do not comprehend it.

I always try to leverage the full potential of the language, but some (not all) of my colleagues perceive that as a risk (some welcome the approach).

I agree it is a problem to write code that nobody else on the team can comprehend. On the other hand we are all professional C++ developers and I think we should aspire to a higher standard than writing C with classes.

My question is, who is right, what should I do?

Clarification: Trying to leverage the full potential of the language, does not mean I throw TMP at every problem. C++ is a toolbox and to me C++ proficiency is about being able to use all the tools from the box and about picking the right one for a particular job.

Answer 1

Metaprogramming is OK. What you are trying to do is not OK.

I use metaprogramming all the time in my job. It's a powerful tool which can be used to do a lot of things in a more readable and maintainable way. It's also one of the harder to comprehend styles of programming out there, so it really needs to earn its keep. I like it when I can reduce 1000 lines of code to 50, but I try to limit it as such.

The issue is not metaprogramming, but this:

On the other hand we are all professional C++ developers and I think we should aspire to a higher standard than writing C with classes.

This is where you get in trouble. You have an opinion. It's fine to have an opinion that metaprogramming is good. It's fine to have an opinion that we should all aspire to be better C++ developers.

It is not fine to compel both your collegues and future hires who will have to maintain the code you wrote to agree with your opinion. That is your boss's job. Your boss is the one who should be concerned with making sure that your code is maintainable in the long run. They (hopefully) have much more buisness experience, because believe me when I say it's a business decision, not an ideological decision.

It is fine to want to metaprogram. It is fine to want to teach others to metaprogram. But understand that it's also fine for others to choose not to learn to metaprogram, and that will be true until you are in a position of power. (and, as an industry secret: when you finally are a lead developer, in a position of power, you're not in a position of power at all. There's someone controlling the pursestrings who is in power).

If you want to encourage them to be okay with metaprogramming, start small. Start with a single enable_if that makes the API easier to read. Then comment the daylights out of it. Then maybe find one case where a template metafunction turns 10 large repetitive classes into 1 class with 10 little helpers. Comment the heck out of it. Get feedback. Find what people think about it. Be fine if they tell you not to do it. Find a niche where metaprogramming earns its keep so thoroughly that your collegues all (begrudgingly) agree that it was the right tool for the job.

As a short story, I wrote a beautiful library once, using extensive metaprogramming. It did exactly what we needed at the time, when no other approach could get remotely close. It actually changed the direction of the application I was writing in. But it was metaprogramming. Only one or two other people in my entire company could read it.

Later, my colleague took another stab at the problem. Instead of leveraging metaprogramming to precisely do what was needed, he worked with leadership to relax the constraints that had been put on the problem such that metaprogramming was not needed. Perhaps more accurately, metaprogramming was less needed. He was able to confine it to what metaprogramming does best.

The resulting library is now in a position to be used in a remarkably wide market, and that's certainly in no small part due to the fact that the new code can be maintained by a far wider range of developers. I'm proud of paving the way with metaprogramming, but it's my colleague's code which is going to reach the wider audience, and there's good reasons for that.

Answer 2

First and foremost, this is the team's issue, and you have to solve it with the team. If you have backup from the team for programming using certain elements and constructs, do it; if not, discuss it with them and if you cannot convince them and make a strong case why "your approach" is clearly better, you will be better off not to use it.

Note that using template meta-programming in C++ is always a trade-off: sure, it can sometimes help to design certain parts of an application more DRY, and it is definitely helpful for creating highly efficient and highly reusable libraries.

On the other hand, these benefits come at a certain cost: the code gets more abstract, often much harder to read, much harder to debug and much harder to maintain. This makes the usefulness of meta-programming in application programming often questionable. So, assuming you are not going to create the next STL, every time you are tempted to use meta-programming, ask yourself if those drawbacks are really worth it. And if you are unsure, discuss this with your peer reviewer during code review.

Answer 3

My general opinion: if you have a choice, as is often the case, between the following three options:

• Type out many nontrivial code structures repetitively by hand;
• Use C++ template metaprogramming to automate code generation;
• Use some other code generation mechanism, such as macros or some other programming language to generate C++ source files

then template metaprogramming, done properly, will likely be the most readable and maintainable of the three options. This is the argument that I would make to the team, if I were in your position. Examples with actual code would help convince them.

When you use Template Meta-Programming(TMP) to avoid repetition, you should use it to construct well-documented, carefully tested abstractions that localize the complexity within the TMP code, making it easy to write correct client code. This is the design of the C++ standard library.

I do not think that we can judge who is right or wrong without seeing an example of the type of code you're trying to write.

Answer 4

Software developers should aspire to write code that works, that works obviously, that can be tested, that can be reviewed, that can be debugged, and that can be adapted when changes are needed. If writing "C with classes" achieves this, then it is just fine.

And these are the standards that you should measure your code against. Especially: Does it work obviously, can it be tested, can it be reviewed, can it be debugged, and can it be adapted when needed?

Answer 5

An argument from compassion:

Does your job give free time for learning, or in alternative, can you convince your bosses to allocate some hours for learning these language features?

If not, using those is essentially giving them extra unpaid work. You may think that a C++ programmer should know the whole language, or something like that, but an academic point doesn't relieve the burden you'll be imposing on them. Your colleagues have kids, ailing parents, sick spouses - or hell, just a reasonable social life that doesn't involve learning C++. The more vocal opponent of your proposal may be lazy - or they may be going thorough a rough time and doesn't need extra shit in their life right now.

Answer 6

Code should be written firstly for humans to read, and only incidentally for the compiler to parse.

Now the thing to remember about non trivial TMP is that you are restricting the number of people who can read that code, it can be a valid tradeoff, but I would argue is far more of a reasonable tradeoff in libraries and such where you have a small specialist team of experts then it is in a bigger application.

When you pull out all the tricks in the book you impose costs on everyone else in that they now need to understand the language including all the lawyerly corner cases that you have exploited, you also impose a cost on hiring in that you have raised the bar to work on the application in a useful way, now maybe that is worth it, but do watch the costs....

For me, a bit more typing, maybe even some code duplication, but that I can put in front of Mr "C with classes plus STL" when it needs modification is a lot more useful then some incredibly elegant TMP thing that only I can maintain (And will therefore be forever maintaining). Remember also that the C with classes guy might happen to be the subject matter expert, and that expertise is usually a lot more valuable then being a language lawyer.

I forget who said it but "Everyone knows that debugging is harder then writing it in the first place, so if you program it as cleverly as you can, how will you ever debug it?".

Even if I can write really out there modern C++ I usually prefer not to, it means I have to do less of the maintenance programming.

Answer 7

No you should not. You are employed to produce code that satisfies a specification. This code has to be maintainable not an ego trip. You could be run over by a bus tomorrow, so someone has to be able to pick up your code and progress the task in hand. However, it is positive to try to convince your employers to incorporate new techniques into their programming standards.

Answer 8

The only way to answer this question in context is to look at the specific problems and the specific way you solved them with meta programming. Unless you post code here we cannot know whether you indulged in an unnecessary complication which is fun for you alone "solving" a non-existing problem; or whether you employed the full power of the language to write a simple, elegant solution not available by other means.

If it is the latter, I would encourage you to continue doing so together with your team. Every good team must do meaningful code reviews which discuss not only style issues but also whether the programer's solution uses the best approach, uses the appropriate language features, is maintainable and testable etc. Your meta programming solution should fill one such review session, likely a whole afternoon. The programmers who reject your approach should lay out alternatives (like code generation with perl, code duplication etc.) and show how it performs against the criteria mentioned, compared to your solution. Your job, as their friendly "opponent" in the argument, is to show that it is a way to get the job done fast, that maintenance is easy, testing is easy, and the code is actually readable once you get past the hurdle of parsing the funny grammar. (If you are strategic you may show it to a friend in the office beforehand and convince him or her that it's really cool; that will help you in the discussion because you won't be alone.)

Most programmers are lazy and enjoy elegant, small solutions. If yours is one, chances are you can convince them, especially if the alternatives demonstrated fall short.

Answer 9

There is not a single correct answer to this question.

Because the first thing you should ask yourself is: in what situation are you placed by your employment? Some people are indeed employed as code monkeys to code up specifications, others are employed as team players, others are employed as knowledge workers or experts.

The only constant bottom line is that you need to look at it from the perspective of your employer, since essentially this means to be an employee. Sometimes it is in the best interest of your employer indeed to nudge your colleagues to become better and master advanced techniques. Yet in a different situation, you might just create lots of problems and liabilities and endanger the success of the projects you're involved in.

Answer 10

The question isn't really about whether meta-programming is OK or not, but rather about whether it's OK to be better than the others on the team so here are a few controversial points about how I see it...

---

I recently moved to a new job where I am working in a larger team and this [meta-programming] worries some of my colleagues, because they do not comprehend it.

They are worried that you're better then them. That's good. You're going to be the new expert. You just destroyed their status-quo world.

---

I always try to leverage the full potential of the language, but some (not all) of my colleagues perceive that as a risk (some welcome the approach).

Sure, nobody likes to be less skilled than anyone else, so they are trying to stop you from using techniques that are too complex for them. They either cannot comprehend it or are not going to, because they feel secure now.

---

I agree it is a problem to write code that nobody else on the team can comprehend.

I don't. I think it's showing your expertise.

---

My question is, who is right, what should I do?

You should use all your skills to write the best code you can write and don't look behind at those who don't understand it. Otherwise you'll get stuck on their level and be just an ordinary coder. It's a good thing to be better than others, and it's a good thing to strive to be better than them. You'll never gain any new experience if you don't try to use anything new or do things differently.

---

I know I'm going to be downvoted, but this is how it looks like. It's not a crime to be better than others on the team, and it's not a crime to use your skills. It's just that everyone is afraid to admit it... because they are on the non-skilled side and hate it that the new guy suddenly can do something they can't. If they were clever, they'd ask you for help and for advice and not criticize your code for being incomprehensible.

---

EDIT

There seems to be a lot of confusion about this question. As the comments show many people think it's about general code readability. No, it's not. It's about whether certain language features/constructs should be forbidden or avoided because some team members don't understand them.

My answer is no. They should not be forbidden. If you want to forbid something how would you do that? You'd have to prepare some kind of a questionnaire to find out what your team members can and cannot - or rather don't want to learn as I think all languague features are useful somewhere so knowing them and being able to use them is always good and the more you know the better code you can write. You'd also need a scale to define which features are beginner, intermediate or advanced ones.

In order to demonstrate how silly such restrictions are, let's take a really simple example: you are going to be hired as a software engineer but your future boss tells you that you will not be allowed to use do/while loops because there are a couple of people on the team that have never used them before and also are not going to because they have always been using for loops for everything so they find do/while loops confusing.

Now you think this is stupid and crazy, don't you? But so is forbidding other features. Some poeple can use them and others don't want to learn them.

Why should you produce worse code if you know there is something that allows you to do the same with much less effort and yet result in a much more readable are robust code?

And it doesn't matter wheter you use only basic language features or advanced ones, you can use either one to produce an equally incomprihensible and unmaintainable code so this is an entirely differen topic.