What arguments to use when explaining to a team that one does not write unit tests in a different language?

Question

I'm helping a software team to embrace a transition to DevOps and I realized that they struggled a lot with testing. The one point that troubles me the most is that they absolutely want to write unit tests in pytest, when the application is developed ... in C#. Part of this excentric idea comes from the fact that they are convinced that it's the QA's job to write unit tests (who don't know anything about C#), but while I do see how to effectively explain that it's the developer's job to write unit tests, I'm struggling to explain to them why they shouldn't try to write unit tests in a different language than the one used to develop the product.

It just seems so obvious to me why doing it any other way would be detrimental to the project that I don't even know where to start. I tried to Google it to find inspiration but I couldn't find anything on that topic. Maybe I'm just plainly wrong, but never have I seen this before, even when I was a developer. And now seems like a good time to address what I think to be a misconception, given that they are only getting started with unit tests.

Which arguments could I use to explain that writing unit tests in a language that's different from the codebase to be tested is a bad idea?

Answer 1

I see two points:

1. It is developers responsibility to write unit test
2. Unit tests need to be written in the same language as the code.

If you succeed at convincing them of the first point unit test become a part of their task.

You first need to understand why they would use python. Maybe it's just because "tests were written by QA in python" or any reason. Understanding those reason will help you advance the proper arguments. You also need to align your definitions of unit test. Maybe they think all kind of tests are the same (hence why they need to be done by QA), maybe their definition of "unit test" is your definition of "integration test". If you don't talk about the same thing you wont understand each others

Once you know that you're all talking about the same thing and I'll go by this definition of unit test (not the best but serve it's purpose)

A unit test is a fast test that can test in isolation a piece of the software code.

Some arguments that can help convince them:

• Developing in one language is simpler than developing with multiple languages. Paradigm, syntaxes, usage, etc. differ a lot between c# and python.
• It simpler to import a unit of C# code in C# code than in python code. The unit may be a function, a class, a package
• It is simple to inject dependencies (same as above)

Answer 2

What these guys are doing is a bit unusual, but not really a problem. At least it makes it unlikely that the unit test just replicates the original source code including bugs, and your unit tests only succeed because they make the exact some mistakes as the code you are testing.

If your C# code can be called from python code without problems, then unit tests written in python should be just fine. There may be problems if the guy writing the unit tests is the only one fluent in python, but that’s not a problem with unit testing.

Answer 3

QAs job is to not think like coders. They should think like obnoxious users. That’s why they write black box tests.

The developers job is to be coders who can read and write code. They write white box tests (and sometimes black box tests).

Black box means you don’t need to read the code. Just test what it does.

QA doesn’t even need to automate their tests. Just think them up. QA should be hammering on the product like the most obnoxious user: entering negative numbers in payment fields, dividing by zero, and clicking on greyed out buttons to ensure they do the nothing they are supposed to do.

The language those tests are first written in is English because, again, these aren’t coders.

But you want all your testing automated. Fine, have the user focused tester sit down with a code focused coder and have them convert the English test into code. That’s still a black box test because it was devised without looking at the code.

Automate those tests and it frees up QA to think of other ways to break the product.

QA is an art form. Don’t disrespect these people by trying to make them into 2nd tier coders.

The coders should be writing the code. They should do it in the language that works for them. Doing that in two languages comes with a cost:

• The C# code will be a black box to anyone that only knows pyTest
• C# only coders will be able to pretend that testing is not their job

And since white box testing is very much a needed thing, and very much not QAs job, no. No this is not a good idea.

White box testing should be done by those that wrote the code. If for no better reason then to punish them for creating code that is hard to test.

Answer 4

You have one idea, unit test should be in the same language as the code, they have one idea, unit test should be written by QA.

Unfortunately, you both are absolutely wrong.

Unit test needn't be in the same language, and in fact there are advantages to writing them in another language; disconnect between the code being tested and the code being executed reinforces the need for the test code to be written with an understanding of what the code under test is supposed to do, and not just copy-pasta. It’s not a requirement and it may add excessive boilerplate code to enable interop between the two as well as demand more skills from the developers, but as an idea, it’s not a bad one.

Unit test should never be written by anyone but developers. A unit test verifies not that the code is correct per some external spec, but rather that the code under test performs as the developer expects and that it’s behavior hasn’t unexpectedly change. A failing unit test is an incredibly useful thing, passing tests are not so much useful as comforting.

Having someone other than developers write and maintain unit tests is at best the equivalent of another code review, at worst it’s like having a jury decide a case based upon a written summary, you get a predetermined outcome that makes you feel good but that’s it.

Answer 5

If you want to build a ship, don’t drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea.

---

I'm struggling to explain to them why they shouldn't try to write unit tests in a different language than the one used to develop the product.

No doubt - you're trying to advocate for constraints; but constraints aren't interesting for themselves, but rather for the properties that they induce.

Nobody's giving our prizes for "unit tests", regardless of which definition of unit test you are using.

The actual benefits are things like being able to reduce the latency between decision and feedback, or being able to obtain the feedback without context shifting, and keeping test execution low, and producing demonstrations of client code in the language that the client code will be written in, and so on, and being able to more readily transfer design insights from the test to the implementation (or vice versa).

---

My guess is that trying to use the label "unit tests" is getting in the way, so you should drop that label, and instead talk about the properties of the thing that you actually want.