8

The testing team (the so-called QA team in some organizations) insists that the development team should share their (the development team's) test cases with them. Their arguments are that the development test cases are the starting point for the QA testing.

As a development team member, I don’t understand the request. Per me, the tester should test the solution based on the requirements. I don’t know if the test team should be shared with the detailed design (low-level design) document. However, we are sharing them the detailed design.

I have read some posts here that say the QA team should share their test cases with the development team for a better solution and for better throughput. But, nothing sort of the development team sharing their test cases with the QA test team.

It looks like my QA team is very happy if I share my development unit and integration test cases and the test results.

  • 6
    QA do not need detailed design documentation. They are supposed to test requirements, not design. – user22815 Nov 25 '14 at 15:15
  • One benefit of sharing this information is the QA team can determine if they are interpreting the requirements the same as the dev team. Or would you rather provide some other documentation or go through it in a meeting? – JeffO Nov 25 '14 at 17:30
  • 2
    Stack Exchange is not a forum... – Peter Mortensen Nov 25 '14 at 18:25
  • 4
    @JeffO actually, that is a good reason for not sharing the development test cases, but requiring to create them directly from the requirements. A major benefit from separate testing function is challenging assumptions made during development by having "fresh eyes" on the requirements. If some QA tests fail because they're interpeting the requirements differently from the dev team - great, that's very important information that would have been missed if they started by reading the "obvious" interpretation made by the dev team. – Peteris Nov 25 '14 at 20:10
19

When your dev team and your QA team don't not talk to each other, there is a certain risk that some tests are unnecessarily done twice, and some others are forgotten. One worst case scenario is when your dev team has implemented some nice automatic integration tests, which run in a few minutes or hours, and your QA people tests the same things manually, spending days or weeks for that task. Another bad scenario is when both groups think that the other group is responsible for some type of tests, and so those tests are omitted.

So, assuming both groups work as a team and not against each other, it makes sense to inform each other in detail which tests are made by which group, and let the two groups coordinate there activities.

  • 3
    in my experience, the most reliable way to inform each other in detail was via test-coverage. No matter if QA or dev forget something, coverage analysis will usually show what went wrong. The only drawback is it's tempting to rely on it too much and turn off brain "we've got 100%, we're perfect!" yeah sure – gnat Nov 25 '14 at 20:53
  • code coverage does not mean the code works. Successful programs are more than just code logic - there's data involved too. Add Environmental factors (eg. works fine on XP, not on Win7) and interaction (eg. I enable the product module and I can no longer log on). Code coverage is developer-centric myopia. – gbjbaanb Nov 26 '14 at 8:44
  • @gnat: coverage analysis maybe a helpful indicator for finding some untested parts in your program which maybe forgotten otherwise, I agree. But of course, to develop a good set of tests, coverage alone is not enough, and it also does not save you from doing the same things unnecessarily twice. And if it is the right tool for communicating things between the QA team and the dev team? This depends very much on how those two teams work. – Doc Brown Nov 26 '14 at 10:16
  • 1
    @gbjbaanb agree about myopia, I've seen way too often people falling into the illusion of "covered=tested", that's pretty harmful. The only useful thing (very useful to be precise) I ever seen from coverage analysis is "gaps", showing what is not covered. Beyond that it simply stops being useful, it can't magically tell if covered code is tested correctly or not – gnat Nov 26 '14 at 10:23
  • 1
    @DocBrown I'd say it's the right tool to control possible gaps in the communication... :) it is in no way a replacement to discussion, teamwork and knowledge transfer – gnat Nov 26 '14 at 10:34
16

Testing team (the so called QA team in some organization) insists that the dev team should share their (Dev team's) test cases with them.

Sure, QA should have a general understanding of what is covered by unit/integration tests, and what is not.

Their arguments are Dev test cases are the starting point for the QA testing.

...even if their reasoning is flawed. The #1 mantra of a QA person is don't trust the developers. "Don't worry about it! I checked that thoroughly!" is the first step towards a giant production issue.

As Doc Brown says, it's not good to spend a whole bunch of QA time on something that is well covered by automated tests. But it is foolhardy to spend no time on something that is well covered by automated tests. And it's wasteful to spend a whole bunch of time documenting your unit tests in detail when QA shouldn't really trust them anyways (and because that level of documentation drives your developers to make less/bad unit tests).

  • I agree fully to what you wrote, the topmost rules of quality assurance are of course "double check everything", and "two pairs of eyes are better than one". – Doc Brown Nov 25 '14 at 16:20
4

In a modern software architecture the intent is that the tests not only test the code but they do so in a way that documents their functionality.

This documentation of what the code does (in addition to what it is intended to do in the specs) is useful for QA'ers to better understand the intent of the code and also whether it matches the requirements. It is also useful information when QA'ers are writing test cases and it will help give them a sense of what is, in theory, being tested already. Some of the areas under test might benefit from additional cases and others might be exposed as not having tests at all.

The actual details of this exposure will depend greatly on the organizational makeup and in particular on the technical depth of the testers (e.g. reading source code).

To be somewhat contrarian... I often like to ignore the developer tests and come up with my 'own' tests (I am a member of a large QE team). I have found that ignoring developer tests helps to avoid the mindset of looking at the problem/issue/feature from a developers point of view only.

MY QE motto is: QE should be adding value by testing the product. "merge and run unit tests" alone is insufficient QA !

3

My first reaction of this question is “it depends”.

It depends on what you mean by “development team test cases”.

  • If you have only unit tests (white-box test); QA Team (integration and system test) could not leverage your test-cases. Unit test are only there to verify your unit, which solely don’t satisfy a requirement (mostly). White-box testing shouldn’t be the way for integration or system tests.

  • If you have behavioral tests; QA Team can use them to derive some integration scenarios.

  • If you are using TDD; as per definition, tests are executable documents of the requirements, architecture and design of the application. QA team would definitely need these test cases.
  • Some test design methods like gray-box testing requires detailed design information to be able to isolate or to mock test subjects like component or subsystem (integration test). Therefore sharing the detailed design with QA team is also necessary.
  • System Test guys never bother with detailed design. They focus on user scenarios. Therefore development tests (white-box) won’t be any use for them.

My humble recommendation for your case is an evaluation of agile approaches like having tester(s) (QA) within the dev-team and design tests (feature, integration, system) together upfront.

1

There's no reason that you shouldn't share what you've tested with the QA team, however they should be duplicating the tests of yours that they feel are important to properly verifying the application.

1) They are responsible for testing the code/application/whatever. By duplicating your tests where appropriate, they verify that your test was done properly.

2) As a developer, you're responsible for unit testing your code (generally, however some organizations have started having dev's test their code as well). This is a different approach, and typically is focused more on covering the decision points in a method.

3) As you mentioned, it's important for your test team to share their test cases with you to help you think of cases that you may not have originally considered.

4) Someone mentioned that it's the test teams responsibility to test to the requirements, and while this is true, the detailed design is quite helpful as well. By having the design, the test team not only can make sure that you're covering the requirements, but helps them to understand some of the design decisions, and will help them to ultimately create better test cases.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.