2

We have a simple CRUD application. QA has a test script where they are doing the following:

  1. Call Create Customer service
  2. Query the database for the newly created Customer
  3. Verify the Customer in the database matches the expected result
  4. Call the Get Customer service
  5. Verify that the Customer returned from the services matches the expected result

My question is: should be QA even be running steps 2 and 3? Or should they be agnostic to how the data is stored and just validate the application output (run steps 1, 4, 5).

On the one hand, it would be good to check that we are saving data correctly to the database (especially if we are saving in calculations, foreign keys, generating timestamps, etc.).

On the other hand, QA shouldn't care how/where we are storing data, as long as it is returned correctly from the application.

  • Who owns the customer data? If you were the owner, would you be happy with Q&A not checking how the data gets stored? – COME FROM Mar 21 '17 at 9:24
  • Who owns the database? Can you make schema changes (e.g. renaming tables/columns) without affecting other applications? – Bart van Ingen Schenau Mar 21 '17 at 9:58
4

In my experience, it isn't unusual for QA to carry out these kinds of tests. The priority of course is to check basic behaviour (black box tests). As an extra sense check, you may want to check the system internals (white box tests). Steps 2 & 3 would fall into the white box testing category.

I don't know how your system is structured but it is possible to imagine a scenario where data is cached in some kind of mid-tier. If the database wasn't checked, tests 1, 4 & 5 would pass with flying colours but if the mid-tier were re-started, those tests could fail. Also, if the test script contained static data to be submitted, you couldn't be sure this wasn't hard coded into the application somewhere.

You can also spot potential sources of bugs by comparing the submitted data to the saved and retrieved data. If the submitted and retrieved data are the same but the saved data in the database is different, it may be worth doing some additional testing to ensure all conversions for the value in question are carried out correctly.

A system I worked on had an issue handling gender. A dropdown in the application had M(ale), F(emale) & U(nknown), but only M, F or space was stored in the database. On further inspection, other parts of the system used a variety of strings to represent blank such as "unspecified", "not known" etc. This is something we wouldn't have found if we hadn't checked the database.

3

On the other hand, QA shouldn't care how/where we are storing data, as long as it is returned correctly from the application.

I strongly disagree.

If what's being tested is the storage of data, then you absolutely have to test that the data is being stored correctly.
You cannot rely on the Application under test to tell you that it's stored the data correctly without some external agent verifying that it has actually done so.

It's like building a brand new thermometer and then trying to calibrate it using nothing but the thermometer itself to do so. You might get lucky and it displays every temperature correctly or, more likely, you'll get burned.

  • This is very true if your system makes use of triggers to modify data not in the original (your 'Customer') table. – Matt Balent Mar 21 '17 at 18:05
  • Agree with Phil. This is especially important when your application data is enterprise data that is being used by other systems and reports. Imagine if your data was used for compliance of Sarbanes-Oxley or for Tax Reporting. – Charles Byrne Oct 11 '17 at 12:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy