4

When Microsoft released Visual Studio 2008, there was a thing they were talking a lot about at the conferences and in their online tutorials: the idea of writing the actual code in one language, and the unit tests in a different language. For instance, unit tests for C# code can be written in Visual Basic.

Six years later, I don't know any open source project which uses this practice and no proprietary project I've seen uses it. The subject is not discussed any longer at Microsoft's conferences, blogs or tutorials.

The benefit of this practice is that doing the same type of error in two different languages is more difficult. For instance, if I don't know that in C#, 10/3 equals 3 and expect it to be 3.333..., a unit test written in Python or JavaScript will fail and lead me to the discovery in integer division.

.NET Framework, specifically, makes it possible to use any .NET-enabled language to test another one. While testing C# code with Visual Basic tests doesn't look particularly interesting, given the similarity of both languages, I would imagine that, for example, testing F# code with unit tests written in C# could be useful for people who are not that familiar with functional programming.

Personally, I have no experience writing unit tests in a different language. Therefore, I don't know if there are actual drawbacks in this technique. I often write system and acceptance tests in a different language (for instance in Python for a Node.js web app, because Python's Requests library is so great), but this doesn't count, since those tests are truly language-agnostic: they just do black box testing through HTTP: I can rewrite an app from Node.js to ASP.NET MVC, and the tests won't even notice the change.

So what could be the drawbacks, and why is this practice not that popular?

  • 3
    I've written integration and module/subsystem tests in a language different than the system itself, but unit tests I consider to be way too coupled with the actual code to use a different language. If a C# function takes an object as a parameter, then the last thing I want to do is to write JS code to mimic said object - I'd rather just switch to a language that makes them naturally. – Ordous Feb 23 '15 at 20:22
9

Unit tests are white-box tests and usually written by the developers of the thing being tested. The first means that you need first-class interaction with the APIs and objects and values of the language the actual product is written in, which rules out almost all combinations of languages. In fact, I think the CLR is the only agglomeration of languages that each can call into each other perfectly. And as you say, most CLR languages are so similar that none of the advantages you suggest apply.

That leaves pretty much only the combination of F# and C#/VB.NET. I can imagine several reasons this hasn't taken off:

  • F# doesn't have that big of a community. Conversely, most C# developers don't know enough F# to even consider this.
  • The people who do know F# are either busy writing both unit tests and actual code in F#, or in a team where they can't use F# at all because nobody else knows it (well). As mentioned before, unit tests are usually not written by a separate team, so people who write C# unit tests for F# code probably still have to know F#. Plus the whole white-box thing: If you don't know a thing about functional programming, even the data structures and APIs exposed by (idiomatic) F# will be unnatural for you once you delve deeper than very simple libraries of the form "put in a URL here, receive IEnumerable<UrlParameter>.
  • There's no benefit (or people don't believe the benefit), but quite possibly some very real friction from switching languages constantly.
  • If you know c#, couldn't you write tests for F# functions written by someone else assuming you know what they're suppose to do and not necessarily how it is done? – JeffO Feb 23 '15 at 20:16
  • @JeffO How often does someone write unit tests for code they haven't written? Other than that, technically yes, but see the entire rest of my answer for why this is unlikely in practice. – user7043 Feb 23 '15 at 20:17
  • Isn't it better to characterize unit tests as black box tests? While they test the innards of an application, they usually treat the part they test as a black box; unit tests written against implementation details are brittle. – Doval Feb 23 '15 at 20:18
  • 3
    @Doval You can use words in whatever way you wish, but merely being aware of (and primarily interacting with) a very specific class/method/function, the existence and behavior of with is only relevant to the people implementing the software as a whole, makes it a white box test in my book and in every classification I've seen. It doesn't need to poke private variables of the unit under test to be concerned with implementation details. – user7043 Feb 23 '15 at 20:21
  • @delnan - If they are doing Test Driven Development (TDD), then all the time. – JeffO Feb 27 '15 at 3:02
3

I do not recollect any such push. I imagine it was a demonstration of interoperability in the .NET space rather than a practical use of different languages for testing. Any benefits would have been made up just to underpin the virtues of this wonderful common language runtime environment.

I guess this could make some sense for library code. If you write and deploy a .NET library that will likely be used from other .NET languages than your own, you may fall into the trap of using features that only work well if both client and server code are the same language, like using language specific types rather than CLR types exclusively.

1

IMO the reason you don’t see this much anymore is because it was really just another way to put a nail in VB. The idea being promoted wasn’t that it’s good to write unit test in another language, but rather that you could write C# unit test for VB code.

This has several potential advantages. First, if your team doesn’t have a lot of c# experience, it’s a way to introduce it to them. Secondly, it’s a foot in the door for future projects. I.e, all of our code is VB, so the next project will be in VB, becomes, the last project (a unit test project, but hey, its still a project) was C#, so it’s OK to make the next project a C# project.

That was in 2008, by 2015, that just wasn’t necessary. Teams still using VB are going to continue using it for legacy reasons, but most new development will be in c#.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.