Why to let / not let developers test their own work

Question

I want to gather some arguments as to why letting a developer testing his/her own work as the last step before the product goes into production is a bad idea, because unfortunately, my place of work sometimes does this (the last time this came up, the argument boiled down to most people being too busy with other things and not having the time to get another person familiar with that part of the program - it's very specialised software).

There are test plans in this case (though not always), but I am very much in favor of making a person who didn't make the changes that are tested actually doing the final testing. So I am asking if you could provide me with a good and solid list of arguments I can bring up the next time this is discussed. Or to provide counter-arguments, in case you think this is perfectly fine especially when there are formal test cases to test.

Answer 1

As others (and yourself) have noted, developers should unit test their own code. However, after that, any nontrivial product should also be tested by independent person(s) (QA department and/or the client herself).

Developers normally work with the developer mindset of "how to make this work?". A good tester is thinking about "how to break this?" - a very different mindset. Unit testing and TDD does teach developers to change hats to some extent, but you shouldn't rely on it. Moreover, as others have noted, there is always a possibility of misunderstanding requirements. Therefore final acceptance tests should be conducted by someone as close to the client as possible.

Answer 2

The developer knows how their code works and will fall into the habit of testing their code according to this knowledge.

The developer will find it difficult to remove themself from the mindset of 'how it works' as opposed to 'how it should work'.

Because of this it is better to get someone with a high degree of objectivity to test the program i.e QA or Test Engineers

Answer 3

Testers Test to break, Simple. This type of bias is needed to really find out the show stoppers.

Answer 4

Developers MUST test their work. It is an implied responsibility.

I'm assuming you don't have a team dedicated for doing the tests based from your statement. However, having a team dedicated for testing will really help as developers tend to test their code the way they coded it. It doesn't mean that once you have a quality assurance team of some sort, you already can take out testing as a responsibility of the developers.

Developers usually use nets with huge holes for catching bugs. As a result, smaller bugs escape.

Answer 5

Because developers are not good at trying to break their own code. Their mind simply follows the correct path of data entry and interaction with the application. Many bugs are the outcome of interacting with the system like a normal guy. Developers are not normal users. They are professional users.

Answer 6

There are a few good reasons to have a dedicated testing team. First, as mentioned above, developers are very good at testing that their code works, but not a breaking it.

Also, as you say, a developer knows what they wrote, but a testing teams knows what should have been written. At times, these two concepts don't match up. One of the jobs of the testing team is to make sure that the software meets the requirements. In many cases, a developer only knows a few parts of the system very well but the QA team knows the whole thing.

Which leads to the next reason, testing teams do full integration testing. The piece of code that you just wrote might work fine on it's own, but could break other functionality that you didn't know about.

Having worked with a QA team and without, I can tell you I 100% appreciate the work that they do and will say they are a valued part of the software team. When you have a QA team, it makes releasing your code that much easier, b/c you know it's been thoroughly tested and that means you'll get less 3am calls.

Answer 7

Developers should unit test their own code.

Independent testers not only test to break, they test the unstated and undefined assumptions that the developers made while coding.

Answer 8

I would expect the developer to do some initial testing before they commit any changes and to have satisfied themselves that the code works. I would then expect the developer to feed into the test cases any specific 'white box' knowledge they have. For instance details any other areas of the code which may have been affected.

The main objection to developers testing their own code is that you are only testing one view point. The developer has read the specification and interpreted it. Hopefully the specification is clear, complete and unambiguous, but this is not always the case. The developer may have misunderstood part of the specification. If they test their own code then this won't be caught as they will find the function operates as they expect.

Different people will also tend to use a product in different way and take different routes through the code as a result. A developer will have ensured that the code works for them, but may not have considered an edge-case that another tester may find.

Answer 9

Developers should test their own work. Letting developers push untested work to a QA team, or their developer colleagues is a Really Bad Idea. It wastes the time of developers and testers alike, and ruins relationships.

However, that is not always enough. Developers are likely to follow a happy path through the system, or be blind to some idiosyncrasies that they have been exposed to over and over throughout development.

Another point is that there can be a number of layers of communication between specification and deployment. This can lead to a Chinese Whispers effect on the final deployable. It is best if whoever defined the requirement or bug report tests that it works the way they wanted.

Answer 10

As a developer you are responsible for your own code, you should test it. Does the feature work as expected? If the answer is yes, you are done.

Why shouldn't you do test-cases?

1. You are subjective, as found bugs are written by you (or your colleagues).
2. You are too expensive for the company to run case-tests. (I hope).

Answer 11

Typically, the developers will not, in most cases, be the ones using the code except in certain specialised cases. So the last testing step before promotion to a production system should be user acceptance testing, UAT. They're generally [supposed to be] more familiar with what they expect the package to be doing. And are generally more capable at breaking things with entry flows unfamiliar to someone who doesn't use it on a day to day basis.

Do your project plans not cater for user testing? If you get users to test it, you may catch bugs earlier than post-implementation which in my world is no bad thing.

Answer 12

Developers shouldn't be testing their own code because that is akin to judging your own child's art. Its gonna look beautiful to you either way, and you really need a professional to point out the flaws. Unit testing on the other hand is akin to ensuring your kid is not trying to paint with lead.

In case you guys REALLY don't want to hire QA, get developers to write test code for other developers. That is a good first step - soon you will see developers asking for QA resources because most of their time gets spent in testing other's code for issues, in addition to CR.

Answer 13

It's not only developers being protective of their code, if they don't realise a particular case, or misinterpret a specification in the way they develop something, then they will miss those cases when testing their code.

The techniques and skills for testing are both very different too.

Most testing by a test team is functional (that a product functions as per a specification) and black-box (the test team will not see the inner workings of an application). Functional testers do not need to be concerned with how things work, they only need focus on whether they do.

Answer 14

In my experience, at least in my small organization, the end user needs to test. Almost every project we get, they fail to provide all needed information, and always leave out certain details. The developer is at always at a testing disadvantage because he does not know how to do the job of the user, so while he knows the software works according to the info he was given, he doesn't know if it will help the end user do their job.

Answer 15

Developers misread and misinterpret requirements and those responsible for the requirements often fail to specify key things. If no one except the developer tests, then no one will find these disconnects before going live. When developers test, they know too much about how it is supposed to work and don't try the stupid things that users might try. Devs also write their tests based on their own interpreation of the requirement which is all too often not what was really meant. So the tests pass but the requirement was not met. When you havea different person testing, that person may have a differnt idea of the requirements and you often find the places where the reuirement was poorly expressed by how differently two differnt people interpret it. Much better to find this out in testing than after you go live.

Answer 16

The Developer should do the initial testing so that we would know the piece which we have coded would work the way it is expected to work, as per the requirements we have got. So we have do the normal testing as well as write Unit Tests for the code we have written.

The next step is the QAs' job to find out what the developers don't see when we write the code. A developer thinks in a higher level but the user might not think in the same level. When developer is testing his piece and has to enter some text in a textbox he might always enter a full string thinking user would also do that. May be user might do it too, but randomly when he enters a special character like %&$^ in the text and that breaks the application it doesn't look good at the end-user. A developer cannot and will not think about all the possibilities that could happen because he is not trained to think that way. When it comes to a QA(tester) they always think about what the user might do to break this application and try every stupid thing in the book, not the users are stupid but we should not leave anything to chance.

Now we also have to understand that there generally more than one piece done at the same time and both will be going to production. The developer could test only his piece and think that is working fine but the overall regression testing needs to be done for all the pieces that are being pushed as well as to find out that the combination of two different pieces could break the application and it does not look good either. We also have to consider the load testing scenarios and other things the testers are more acquainted of.

Finally we have to go through UAT(User Acceptance Test) to see if the piece we did is what that is expected. Generally though the requirements get through BAs the final person might not exactly know how it looks like and he/she might think its not what they expected or they might want to add something else to make it look better or for some reason they might scrap the whole piece as they think the piece would not go with the already functionality available.

As explained above these are very important and cannot be done by the developer alone and are absolutely needed for the application to work fine. The management can say this is a conservative approach but it is the better approach. We can make some tweaks to the above said but cannot avoid as of whole.

Answer 17

The comments above raise great points.

An additional one not previously mentioned is that having a separate individual test code acts as an additional check on the requirements, and if the system correctly implements them.

Requirements and documentation are not perfect, and often implementation is a result of the interpretation of requirements by a developer.

When testing is done by a separate individual, they also provide their own interpretation of requirements when creating the test plan and executing the tests.

When the testing activities are done independently from the development activities, and the outputs of both "agree" it provides an additional confirmation that the system is correct and truly matches the original intention of the requirements.

Answer 18

A programmer, when testing, will see a text box labeled "Quantity" and enter "1". A highly experienced programmer will then do a followup test with the value "2".

A user will see a text box labeled "Quantity" and enter "~~ unicorns ROX !!! ~~". An experienced user will also try "-12 1/2".

Hopefully, a tester will be in there somewhere to alert the programmer about what the user is going to experience when they do these things.

Answer 19

One reason is because developers are too close to their own code. They know it's quirks, it's little weird behaviours. They tend to test around the little idiosyncrasies they know so well. They are not objective enough about it. Test teams treat it like a black box. They write matrices of dozens or hundreds test cases and methodically run through them to see what the code will do. Often they come up with scenarios the development team would never dream of.

Another reason is time. For large projects that are built in stages, the development team will build Stage 1. Then the testers will test it while Stage 2 is being built and the defects of Stage 1 are fixed. This goes on for all stages, so stage being tested is the previous one that was built.

Answer 20

I want to gather some arguments as to why letting a developer testing his/her own work as the last step before the test goes into production is a bad idea, because unfortunately, my place of work sometimes does this (the last time this came up, the argument boiled down to most people being too busy with other things and not having the time to get another person familiar with that part of the program - it's very specialised software).

Tests aren't optional for a developer. A developer has to test the code he wrote. How else can he be certain, that the task has been accomplished successfully? You either have to write some kind of automates tests (unittests) or perform the job of checking "is the machine doing what I want it to do" manuall (by using the GUI, calling the command on the command line or whatever).

Everything that is being tested after that is "only" additional testing by other people (coworkers, QA, ...). There is no alternative to direct testing by a developer. Everyone who tells me that a developer doesn't have to test (or even isn't allowed to) the code/feature he wrote simply has zero understanding of how software is being developed.

Answer 21

It gets tested by someone unfamiliar with the code whether you like it or not. The question is whether you want that someone to be your customer.

Answer 22

Great question. In your situation, test cases -sometimes- exist and the software seems to be complex enough that getting a novice up to speed on the product isn't practical. You also say the test performed is the final test prior to production

Reasons it might be okay for the developer to do the final test

• There is enough other test coverage... Unit tests exist, an integration test environment exists and is used, UI testing, exploratory testing, etc. has been performed, etc. Then a final test is less a rigorous acceptance criteria than a final "run through"
• A set of test cases written by a professional SQA/Tester exist that someone (a developer) can/needs to follow explicitly
• The risk of failure of the feature/product/module has otherwise been mitigated to low levels (let the professional test the high risk areas, and a "rookie" test the lower risk)
• The reality of the business situation is that releasing a product with potential defects is better than delaying the release
• The developer in question is actually a very qualified tester as well and is able to mentally make the change in roles
• The change is a bug fix made in the field by the developer when the customer's site is shut down or otherwise losing revenue because of the system being offline (a patch which will be brought back to the office and tested/released in a controlled version ASAP)

Reasons a developer should not do the testing

• Anything else

In general, it seems like you are on the right path of attacking the real solution - Have the SQA expert generate the Test Cases...

Note: I am generally in favor of letting Developers do the testing, but I make damn sure the first bullet point exists....

Answer 23

Human beings, being human, tend to suffer from Cognitive Bias - where their judgement in two nearly identical scenarios will differ, simply because of a few things that have changed - one thing I have noticed in 8 years of development, is that when a developer is faced with testing his own code, as opposed to code that a colleague has written, the testing performed on their own code is alot worse quality.

This isn't to say that the developer is at fault directly - his/her brain will use the bias that they wrote it, to reinforce the fact that they believe its right, and will only perform basic checks, as opposed to a developer who is looking at someone else's code, will do alot more thorough checks.

There are thousands of examples out there where procedure has been put into place to prevent cognitive bias, or commonly known as "The Human Factor", such as computerised systems in air traffic control, to prevent two different aircraft occupying the same airspace at the same time, to medical procedures put in place so more than one doctor have to give a diagnosis.

Its about time that the IT industry moves towards a more professional attitude and puts in procedures in place to prevent testing of your own code.

Answer 24

• Eeveryone should test - Develpers test code, QA'ers test functionality, Marketing test messaging. That way everyone shares the same philosophies and language around testing which is half the battle.

• Testing is routine maintenance and I usually use analogies to compare. For instance the car oil change analogy. You never 'have to' change your oil. But you do it regularly anyway. Same for brushing your teeth. There's a reason you maintain them daily - they're not going to break 'today', it's all about tomorrow and future days and making an investment.

• Everyone should share in the responsibility to test. A QA team is important, however making "testing" as something that only the QA team does makes it to be a 'separate' activity that it not integrated to product development and workflow which is not a good thing.

• When anything ever breaks in production do two things:

  1. Say "hmm, do we have a test for that" as the first comment.
  2. Make any fix include tests for the issue, first to reproduce, than fix.

Answer 25

At my company, we build some pretty complex financial applications. Our general policy is that the developer should ensure that no technical errors arise. Basically, try everything you can to break it, given the resources of the user. When you cannot find a runtime error, send it forward to the BAs for testing. We've had a few developers who got lost in testing business requirements to the point of burn out, but only because all that testing was not their responsibility. Unless there is some glaring error that is clearly visible, we send it forward to the people who get paid to understand the output. Also, users should have a real role in verifying the results. The sales clerk at a retail store doesn't try on the clothes for you, they only help you with the "technical details" like finding clothes of the right size.

Answer 26

One issue is developers have little incentive to break their own code -- few people are willing to search for defects in their own works or are willing to make mistakes. Having a separate team helps ensure that things will be broken.

Answer 27

A Quality Assurance role is essential, amongst other reasons, so that someone can check that the developer has understood the requirements. The developer cannot do this check themselves because if they thought they had misunderstood that would ask for clarification.