5

ISTQB, Wikipedia or other sources classify verification acitivities (reviews etc.) as a static testing, yet other do not. If we can say that peer reviews and inspections are actually a kind of a testing, then a lot of standards do not make sense (consider e.g. ISO which say that validation is done by testing, while verification by checking of work products) - it should at least say dynamic testing for validation, shouldn't it? I am completing master thesis dealing with QA and I must admit that I have never seen worse and more ambiguous and contradicting literature than in this field :/ Do you think (and if so, why) that static testing is a good and justifiable term or should we stick to testing and static checks/analysis?

Wikipedia: Static testing is a form of software testing where the software isn't actually used. This is in contrast to dynamic testing. It is generally not detailed testing, but checks mainly for the sanity of the code, algorithm, or document. It is primarily syntax checking of the code and/or manually reviewing the code or document to find errors. This type of testing can be used by the developer who wrote the code, in isolation. Code reviews, inspections and walkthroughs are also used.

EDIT: One of the many sources that mentiones static testing to employ reviews etc. (look for "static testing" in Google books):

enter image description here

2
  • 2
    It's QA, not testing.
    – haylem
    Commented Nov 13, 2012 at 12:55
  • 3
    ISTQB® is the International Software Testing Qualifications Board, for those like me not familiar with the acronym
    – neontapir
    Commented Nov 13, 2012 at 16:40

3 Answers 3

Reset to default
8

I've never heard of the term "static testing". I have only heard of the term "static analysis", which refers to any time a work product is examined without being used. This includes code reviews as well as using tools such as lint, FindBugs, PMD, and FxCop.

Here is some information from sources that I have available:

  • Section 5 (Software Testing) of the Guide to the Software Engineering Body of Knowledge does not reference "static testing". It refers to "static software quality management techniques" described in Section 11 (Software Quality). Section 11 only has a single mention of tools being used to support these activities, stating that the analytic static activities may be conducted "with or without the assistance of tools".
  • Ian Sommerville's Software Engineering, 8th Edition's glossary specifically mentions static analysis as being a "tool-based analysis of a program's source code to discover errors and anomalies". However, in Chapter 22.3, Sommerville does refer to inspections as "one form of static analysis". There is no reference to "static testing". This book is considered one of the canonical references about the breadth of software engineering and is often cited as recommended reading before both IEEE certification exams.
  • Roger S. Pressman's Software Engineering: A Practitioner's Approach, 6th Edition makes no references to static analysis or static testing that I could find in the index or the chapters on testing.
  • Steve McConnell's Code Complete, 2nd Edition makes no specific references to either static analysis or static testing. However, Chapter 30.2 has a section about analyzing code quality. Tools to automatically check syntax and semantics are called tools that "examine the static source code to assess its quality". A specific example is Lint, which is frequently referred to as a static analysis tool by other sources.

Analysis tools can only be used to verify the product. Human reviews of artifacts can be used to perform both verification and validation. Testing that involves the software's execution can be verification, validation, or both, depending on the context. The key difference is that verification is concerned with finding mistakes and defects. In contrast, validation is concerned with ensuring the requirements adequately describe the customer/user's needs, and the work artifacts (design, implementation, and tests) correspond to the requirements (and the products that they are derived from).

8
  • Thomas Owens: Added reference. Actually nearly every book I find on google books mentioned this. I dont agree with that either. In addition, according to ISO 12207, testing is only validation activity, making sure requirements for use were fulfilled.
    – John V
    Commented Nov 13, 2012 at 13:10
  • I thnk the V&V meaning is different, it depends on the standards and approach. I stick to ISO so verification is just making sure requirements are followed in design, code, integration etc., but it is not testing. Also defects would be fond during validation, which is actually checking if it works as expected.
    – John V
    Commented Nov 13, 2012 at 13:17
  • @user970696 I'm not familiar with ISO 12207. My experiences are in CMMI. However, I question that definition from a practical standpoint. User acceptance testing is a validation activity, however unit testing and integration testing are generally considered verification and validation activities.
    – Thomas Owens ♦
    Commented Nov 13, 2012 at 13:40
  • Thomas Owens: the thing is that ISO considers all testing as validation because you make sure that requirements for use are fulfilled, whereas in verification you make sure that all products are developed accorfding to their specs. I agree ISO is not clear enough, yet unit dynamic unit test would be considered to be a validation, because you are checking that verified code works as expected. A good example of what I mean is here: testingexcellence.com/v-model
    – John V
    Commented Nov 13, 2012 at 14:51
  • @user970696 Like I said, I'm not familiar with the ISO standard and to the best of my knowledge it's not freely available for me to read. I'm providing my knowledge, based on IEEE publications, the CMMI, and commonly used and accepted resources to answer your questions. I answered your question about the term "static testing" - many reputable and frequently cited sources and standards don't use it. If the ISO does and you are basing your work on their standards, then you should use their terminology. However, to many practitioners, it's not a meaningful term. Pick a standard and use it.
    – Thomas Owens ♦
    Commented Nov 13, 2012 at 15:04
2

It's (S)QA, not Testing!

It doesn't actually perform any testing, so I'd be surprised why this would be treated and refered to as such. Except if you expect the reviewers to test the program along with their review of the code.

8
  • haylem: It is a matter of interpretation as static analysis can be well Quality Control - it does not assure quality, just measures how good the documents/designs etc. are.
    – John V
    Commented Nov 13, 2012 at 13:34
  • Maybe I'm not understanding, but when you do a code review, you don't actually "run" the code in your head? You don't try to find problems with the algorithms? Or if you do, how is that not testing it?
    – Caleb Huitt - cjhuitt
    Commented Nov 13, 2012 at 18:14
  • @CalebHuitt-cjhuitt: Of course I do, but it's not testing. You're not actually running the program, you're neither attempting to run a piece of functionality or the product itself. It doesn't verify anything. It's not a validation or verification, it's merely what it says on the tin box: a review. But I'd advocate that reviews should in most cases be done by a developer who also tests the code, and verifies that any unit-tests are working, and also review the unit tests that go along with them. It would be warranted to call this whole thing "testing", obviously. Hope that clarifies.
    – haylem
    Commented Nov 13, 2012 at 18:22
  • @haylem: Actually, according to the literature, static techniques are verification, because reviewing, inspecting is a way to verify that the product conforms to its specification. Verification is (e.g. by ISO) done through review, walkthroughs, inspections. You said it, its QA and QA is - by many, considered to be set of procesess we are talking about.
    – John V
    Commented Nov 13, 2012 at 20:00
  • @user970696: hmm, 2 things: 1) yes, I know I am wrong in saying review is not verification - it formally is, if your review is in accordance with your audit or certification body's requirements (and many, many of them are rather moronic, but sure, it's verification alright) 2) I entirely agree with that whole set of processes being QA. Exactly. And testing is just one of them, with reviews, and reviews aren't testing. That's on that last bit that we don't seem to agree.
    – haylem
    Commented Nov 13, 2012 at 20:10
1

Code reviews nowadays may, on occasion, pick up bugs. But they are more about reviewing the quality of the code rather than the quality of the product.

But there was a time when static testing -- stepping through the code with an eye to catching bugs -- was a justifiable technique. That time was when compilation was something you put in a VMS queue and waited four hours for it to run. Then you'd set up a test run and put it at the back of the same queue. Or, earlier, you'd print the code out on cards and carry them over to an operations room, expecting to be able to test it in a couple of days.

Spending an hour running through the code before (or, often, while) waiting for that was a very good idea.

Now you can compile in seconds and run instantly, so the benefits of static testing have long gone. But those days were less than 20 years ago, for some of us, and that might be why you're still seeing reference to code reviews as an application QA activity.

1
  • ...I can't agree that code reviews are not a QA activity. I'd say that our CR's prevent ~50% of my team's defects from escaping to production. Of course, an automated test suite is a great thing to have, but still not a replacement for a 2nd set of eyes.
    – RubberDuck
    Commented Dec 11, 2016 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.