I've just came across an interesting collaborative coding issue at work.

I've written some unit/functional/integration tests and implemented new functionality into application that's got ~20 developers working on it. All tests passed and I checked in the code. Next day I updated my project and noticed (by chance) that some of my test methods were deleted by other developers (merging problems on their end). New application code was not touched.

How can I detect such problem automatically? I mean, I write tests to automatically check that my code still works (or was not deleted), how do I do the same for tests?

We're using Java, JUnit, Selenium, SVN and Hudson CI if it matters.

  • 1
    I'm not even sure how you'd "accidentally" delete whole swaths of code if you're actually doing a proper pull -> merge -> commit thing. – Anon. Feb 4 '11 at 2:59
  • @Anon I'm note sure either, he says he was in rush and needed to commit his code quickly, so he didn't pay much attention to merging thing or smth. :-/ Anyway I still want to detect such problems automatically on CI level. – parxier Feb 4 '11 at 3:04
  • 10
    And the person "who was in a hurry" might need a quiet talk from a manager, such behavior is lazy and should not be acceptable. – quickly_now Feb 4 '11 at 7:44
  • 1
    I can only imagine this would be possible if people are checking in huge changes with lots of files modified over a very long time. You shouldn't normally have huge merges where there's even a possibility for code to be "lost"... that sounds like the real source of the problem to me. – Dean Harding Feb 4 '11 at 8:46
  • 1
    This is why an individual developer should never be allowed to merge to trunk in centralized VCSes. Lazy devs have a tendency to clobber other people's stuff (been guilty of it myself). – Chris K Feb 4 '11 at 21:03

I'm not super-familiar with Hudson for CI, but my CI tool can also compute code coverage. If you can write a process that will notify you when code coverage goes down, that would be a good indicator that a test has been removed. It would also tell you if new code has been added without tests. Not what you were asking about, but nice to know.

  • 3
    I was about to mention this very point in a comment on Tim's answer: your code coverage percentage should never decrease. – Frank Shearar Feb 4 '11 at 8:54
  • Good angle on a metric! – user1249 Feb 4 '11 at 13:45
  • What tool is that, perchance? – Chris K Feb 4 '11 at 21:01
  • @Chris, we use TFS + TeamBuild, which I have configured to calculate code coverage on each Build. – Marcie Feb 4 '11 at 21:12
  • It won't work on this particular project because test coverage is quite low at the moment anyway, so I have to try Tim's idea. But you gave me a good solution and I think it is best answer to my question. – parxier Feb 5 '11 at 4:17

Standard disclaimers apply: we're making an engineering solution to a social problem. However, this is a project hygiene issue, so it's a bit like saying toilets are an engineering solution to a social problem.

Have a job hand off the RSS feed from Hudson. Count the number of tests in the Hudson report. If it diminishes, sound an alarm. Have an auto-da-fe' when the alarm sounds.

The blamee of the commit can be identified and punished. Your problem will go away.

You might make other problems as a result of this solution. If dizziness persists, please see your doctor.

  • 1
    +1: "we're making an engineering solution to a social problem". That should be the end of the answer. The rest of the answer is less valuable than that one statement. – S.Lott Feb 4 '11 at 3:35
  • 2
    @SLott yes, but if you make it easy to do the right thing, it will get done. We have used an automatic email to the whole team, that was triggered by breaking the build. It works; you get more careful. I personally doubt the usefulness of trying to solve this social problem. If you honestly thing deleting tests is okay, then the company culture is against quality. – Tim Williscroft Feb 4 '11 at 3:39
  • Wow, I'm Portuguese, and I had no idea what an auto-da-fé is. – R. Martinho Fernandes Feb 4 '11 at 5:31
  • The test count can diminish for valid reasons: a refactoring could remove a class and all its unit tests. However, it's still worth finding out why the test count dropped. – Frank Shearar Feb 4 '11 at 8:53
  • @Frank I suppose what would matter if how often the test count drops for valid reasons compared to invalid ones. If it's mostly valid reasons, then the alarm will just get ignored after a bit and become worthless. If it's mostly invalid ones then it could be good. How often does this happen? And actually, @parxier, if it's only happened once that you know of, might you be over-reacting? – James Feb 4 '11 at 9:35

Organisational approach

Have a policy in place which would require person deleting tests to talk to the test creator. Normally you would delete tests only when depreciating some functionality being tested, and that does not happen very often.

Technical approach

This is more of the control freak approach but you can have a separate test, which scans the source code for the presence of all tests you want to check. Possibly you could also interface Hudson and get the list of executed tests.

  • Stuff was deleted during a merge. Maybe by accident, maybe by laziness. Policy isn't going to do much apart from more "thou shalt" from management which everyone will ignore. A public burning and flogging might get some attention, though. /sarcoff – quickly_now Feb 4 '11 at 7:47
  • 2
    @quickly_now: "Stuff was deleted during a merge". That should be a firing offense. Any organization that allows this behavior really needs to remove a lot of people and replace them with folks who make an effort to do something sensible instead of evil. – S.Lott Feb 4 '11 at 12:09
  • Accident - you can forgive it the first time. Laziness or malicious - yes - sacking offence. – quickly_now Feb 5 '11 at 0:47
  • It would be hard to keep that separate test class up-to-date, but it's an interesting idea, thanks. – parxier Feb 5 '11 at 4:20

Similar to Art's answer..

Comments Start by using commenting well. For each method; don't forget to place expected input and output, a short description for more complex functions and your name.

Guidelines But this really highlights that there is a need for more communication between the dev. team. There should be guidelines in place for working together... or at least talk to your proj. manager and ask him to clarify this among the team.

Proper SVN usage You could also write down your classes and methods and track them.. also as you are using SVN I would sincerely hope that these deletions are being tracked as changes, noted separately and have GOOD reasons.

Short of writing a special program, you could also just compare diff. files in SVN to track changes to your methods.


The same could happen for actual code too, and you won't know until you notice your change no longer exists.

That being said, it's difficult to identify code being removed as being a bad thing, as very often you do manually remove code/features etc, and because of that test count may go down as well as someone else mentioned.

  • When code is deleted, tests break. When tests are deleted nothing breaks. – parxier Feb 5 '11 at 4:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.