0

I spend most of my time analysing data and developing machine learning algorithms with data. As a result, I spend most of my time coding but I never have fixed specifications. Rather, I start with a question I want to answer which leads me to the next question, and so on.

For code that won't change, I generally start by writing the tests and have enjoyed this approach. However, I do not know how to use TDD in a research setting.

Is there a way to use TDD for code that gets run only once before being modified further? (Probably not judging from this: Q1 Q2)

What I like about TDD is that it helps me write modular code that makes it easier to reuse code.
Are there approaches that improve code quality in the described setting with changing specifications?

3

I've found when developing research code (graph algorithms and sparse linear algebra for graduate work and in a corporate research lab), unit tests end up being invaluable. When your algorithm performs worse than you expected, you need to be able to determine with confidence that it is an issue with the algorithm and not the implementation. If you don't do this, you may end up discarding promising research or worse publishing incorrect results.

Test-driven development is for continually changing and incomplete specifications. By making sure your code works a the low-levels, you can re-organize it, change it, and re-use it. I've seen many graduate students throw away many months of work because they tried to save time not verifying that their code works.

Anytime you identify a function you want to write that has defined behavior (e.g., norm2(), calculateHitRate(), calculateResidual, etc.), write a unit test for it. Some of your unit tests may be of the form of sanity-checks rather than actual output validation, such as ensuring a matrix is symmetric or semi-positive definite. Basically make sure at each step of your algorithm, your data has the properties you expect. Remember that in TDD, writing the minimal code to pass the test does not imply to write incorrect code that passes the test, but rather to not write features that are not tested.

In research I think automated acceptance tests have much lower value, as you will be doing qualitative analysis of your results in many cases (figuring out why your algorithm preformed worse or better). If your algorithm results in a hit rate of 0.21 instead of 0.24, how will you know it is due to a bug, or just that your algorithm is not as good on that dataset as you hoped?

1

You can apply TDD at higher levels than unit tests. Specifically, you can form your research questions into high level acceptance tests. These tests will describe what successful research will look like. Write a failing acceptance test, and then iterate over research and smaller TDD cycles until your acceptance test passes. Then write another failing acceptance test.

enter image description here

You don't necessarily have to use TDD at a lower level (or at all). Investigate the advantages of TDD, and decide whether they balance the costs. If your research code is not going to be incorporated into production software without being rewritten, then it may be a lot of wasted effort to write unit tests. Your organization may value speed of change over stability in which case having lots of unit tests will limit you.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.