3

I'm working as a programmer in a load cell manufacturing company.

We develop our own software for load cell manufacturing and testing purposes.

Quality Assurance Department of our company has requested to perform periodical verification and validation tests (once a year) for our released software which has been already tested. (They are considering software as a equipment such as a multi-meters which requires periodical verification.)

My opinion is there is no point of testing a released software from time to time.

So I'm here to find answers for the following questions from some experts:

  • Is it a common practice to perform validation or verification tests time to time for industrial software which is already tested and released?
  • Whether it is a common practice or not, are there any supporting reasons to do so?
8

It does not seem to make much sense to run the exactly same set of tests against the exactly same, unchanged piece of software once a year, as long as the software runs on a reliable, unchanged hardware, in an unchanged OS environment, and the program behaves deterministically in context of the tests.

However, if your company is not just testing the software, but the system as a whole (including hardware, operating system, sensors, production controllers etc.), or if the tests got improved in between, it can make sense. And the fact your software was once tested and released does not make it "bug-free", it just proves it contains only undetected bugs so far.

In your case it sounds your QA department just wants the tests to be repeated though nothing has changed only for formal reasons. Well, if they insist, make sure you have all your tests automated, run the test suite once a year, send them the test protocol and make sure you bill them for this.

  • 100% agreed. :) As you said "unchanged hardware", what if the computer is replaced? Do we have to test for each new computer? – Sherantha Feb 23 '17 at 6:45
  • 2
    @Sherantha: depends on what that 'computer' is. If its just something obtained from e.g. Dell, the replacement could have different software on it thus warranting the tests. – whatsisname Feb 23 '17 at 7:08
3

In addition @DocBrown's answer, another thing to consider is if the software runs in a set environment, or if the environment is under user control. So, for example, if the software is embedded in a larger product in which you control the boot-image of the system (e.g. Linux or Windows or macOS), and the user has no ability to update the OS environment, then it seems rather pointless to re-test something that hasn't changed at all.

OTOH, if your system does things like applying OS updates (security or otherwise) or if the user has control over the OS environment that your product runs within, then it is certainly possible for new issues to appear in the overall system, even if your software has not changed (e.g. the OS updates are a source of entropy).

So in the latter case, even if you don't have a new release of the software, it may make sense to periodically run your test suite against the software installed on a fully up-to-date system. If new problems occur, then you can either fix them, or declare that configuration as non-supported.

1

There is an important testing process in the ISO quality standards world that reads like, "Is the software that you are shipping today the same software is what you delivered and we tested yesterday?"

The short answer for many types of software is determine the identity of the software as a complete package by comparing the check sums. So, deliver the check sums of the software as part of the release package, and get a sign-off on that software. That sign-off can be reverified by calculating the check sum again.

Naturally, if any part of the environment where that software runs has changed, the entire system will need to be retested, including the changed environment. Anything else is busy work.

  • It's really helpful and informative. Thanks a lot. If you can please provide a link to the documentation of the quality standards (ISO or any other) you mentioned where I can get familiar with those. – Sherantha Feb 24 '17 at 4:44
  • There are a lot of resources around the net, and quite a few consultants, concerning ISO quality standards. If your QA department is asking for the tests, I presume that they are trying to work to a standard, so that is what you must comply with. – BobDalgleish Feb 24 '17 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.