Sorry that this answer is so long, but your points are well constructed and require details and nuance to counter. If you're looking for a TL;DR, I think the last section is the most important one for your specific circumstance.
Testing your own work is prone to bias
Never in my experience has a unit test failed because the unit under test wasn't doing what the unit under test was contracted to do.
Who is writing the unit test? The developer who wrote the implementation? Yeah; then that's exactly what you're going to get.
Let's say I'm asked to implement a factorial calculation (!
) but I misunderstand the requirement and think its about addition (6! = 6+5+4+3+2+1
) instead of multiplication (6! = 6*5*4*3*2*1
), then I'll obviously write a wrong implementation. But if I design the test as well, then I'm going to write tests that confirm that the resulting value of 6!
is 21
.
The only thing a dev can catch when they write their own tests are issues where they mistakenly wrote code that does not do what they think it does. But anecdotally I can tell you that this is not how most bugs originate. It is far more common for developers to make misinformed decisions, whether that is due to a misunderstanding of the requirements or of good practice guidelines.
A significantly more reliable way to catch misinformed decisions is by having the test cases designed by a different person than who designed the implementation. Regardless of who is wrong, if the tests disagree with the implementation, this alerts the dev (and tester) to find out why they're disagreeing.
The danger of regression
I'm unaware of any instance of a unit test ever failing for any reason other than because the developer changed the unit under test without changing the unit test. So, in essence, the unit test discovered that it wasn't in sync with the code it was testing, but that's all.
I already addressed that this is inevitably the only thing your tests tend to do when your developers write their own tests. However, I also want to address the undertone of your question that asserts that this behavior is not valuable, because I am of the opinion that is very valuable.
The necessity of tests is only visible when you are suffering the consequences of not having any. I've worked on legacy codebases that were a game of non-stop whack-a-mole. Whenever you fixed any issue, you'd invariably create another issue somewhere else. Every time I made a change, I would live in fear of what else was going to break because of it. This is an incredibly stressful environment to work in.
What I'm describing here is exactly the same as when you say:
the developer changed the unit under test
The benefit that you have, which I didn't have in those past projects, is that you have tests to tell you exactly what you changed. Without that test to alert you, if you cause something to change that you weren't aware of, you wouldn't be made aware of it.
Tests flag issues when they occur, and they prevent issues from being unseen and piling up until you get to a tipping point where there's so many issues that it becomes impossible to tackle them without creating new ones. Therefore, the existence of tests completely hides the problematic consequence from not having tests.
You've already been referred to the preparedness paradox, and this is exactly what you're succumbing to. You're already sitting on the greenest grass and are therefore subconsciously not valuing the work that went into making sure you'd be sitting on greener grass.
Better to fix things that prevent them??
the hours spent creating the tests is time that could better be spent on fixing known bugs, refactoring code, addressing technical debt etc.
Strong disagree. I cannot overstate how much I disagree and have personal anecdotal evidence to back this up.
I think you're making a biased observation from a position of a reasonably good developer experience - whether this is because of high developer skill, focus on good practice, decent software development budget, or strong test coverage; is unclear to me. Regardless, I can count at several projects I've worked on in the past where bugs that should be trivial would take months to actually fix.
I started at a company where they said "this project is pretty much done, 5 more bugs and it's delivered". The requirements never changed, but 18 months later the project was still "almost done", now with 21 bugs, and two senior consultants had already been brought in to do major refactors.
In hindsight, it would have been faster to throw out the entire codebase and start from scratch (from the point where they hired more experienced developers to lead the dev team). This is how bad things can get.
This is why I can tell you hand on heart that the pre-emptive effort of writing tests feels like a lot of effort but that's because you don't see what kind of future headaches you will have prevented.
If you were to create two timelines, one where you test (A) and one where you don't (B), and then measure the time it took to write the tests (A) and the time difference in bugfixing (B minus A) to get the application to the same level of quality, B will have taken more time by several factors.
I am not a time lord and can therefore not show you the same project in both of these scenarios, but I have over a decade's worth of anecdotal experience as a consultant (who was often brought in to projects on the brink of failure to fix things) to back up my assertions here.
The difficulty of writing tests
My other gripe is that when I have to write them it seems that frigging the setup (mocking repos, interfaces etc.) is just as likely to be error-prone as the test itself
I'm ignoring possible explanations like low developer skill or lack of testing experience as those issues have trivial solutions (education, experience, mentoring).
You should use this as an early indicator that your contracts between your components are too contrived.
If you think of your application space as a blob of logic, and we are in the business of subdividing that blob into separate components, there are countless ways of making those divisions.
The better way of dividing things is so that the components themselves contain the complexity, and the contracts (i.e. what links the components) are as simple (i.e. not complex) as possible.
If you're in a situation where that's inverted, where the components are simple and their contracts are complex, your argumentation makes sense:
- Implementations are trivial, what's the value in testing them?
- Contracts are complex, tests are complex to write.
This causes friction for two reasons as mentioned above. It is of course possible that only one of these points is relevant to you, either because both your implementations and contracts are trivial, or because both your implementation and contracts are complex. I can't decide that for you.
I maintain that testing is beneficial to product quality. However, as mentioned in the above points, certain scenarios are easier and more justifiable to test than others. This shifts your responsibility. Not only should you be testing, you should also be designing your code to be as efficiently testable as possible, to ensure that you can at the same time reap the benefits of testing your code and still produce implementations are an effective pace.