2

I am currently building a test plan for the system I am working on. The plan is 5,000 lines long and about 10 years old. The structure is like this:

1. test title
   precondition: some W needs to be set up, X needs to be completed
   action: do some Y
   postcondition: message saying Z is displayed
2. ...

What is this type of testing called ? Is it useful ?

It isn't automated.. the tests would have to be handed to some unlucky person to run through and then the results would have to be given to development. It doesn't seem efficient. Is it worth modernising this method of testing (removing tests for removed features, updating tests where different postconditions happen, ...) or would a whole different approach be more appropriate ? We plan to start unit tests but the software requires so much work to actually get 'units' to test - there are no units at present ! Thank you.

Improve this question
4
  • this looks like functional or maybe exploratory testing. Whether it is worth automation or not, depends on details you didn't provide - I've seen both successes and failures of automating stuff like that
    – gnat
    Sep 13, 2012 at 13:40
  • Thanks for the link. What are the details I didn't provide ? I would be happy to give them.
    – ale
    Sep 13, 2012 at 13:43
  • details hm. What technology is under test, how stable and detailed are product requirements, what is anticipated frequency of functionality changes... I may have missed something in this list
    – gnat
    Sep 13, 2012 at 13:49
  • 1
    This style of testing, with precondition-action-postcondition triples has its roots in Hoare Logic. It is a so-called Hoare-Triple.
    – MattDavey
    Sep 13, 2012 at 18:52

5 Answers 5

Reset to default
11

From what you write, it looks like a list of test cases for manual integration testing / GUI testing. It might not be efficient, but any tests are better than no tests and any plan is better than no plan, so you might as well start with what you have.

Certainly you would need to review the plan and update it to cover changes made to the software over time. Obviously having obsolete test cases for nonexistant functionality would be counterproductive.

Keep in mind that even though this test plan was written with manual testing in mind, you might as well automate the process. Research automated testing tools available for your project/platform. If it's a web application, look into Selenium. There are solutions for desktop applications too, but I'm not familiar with any to make a suggestion. Even though it may be a time consuming task, automating these tests, you may find it easier then introducing unit tests. You may also find it useful to have these automated tests when you get around to refactoring the project, as you can use them to validate your changes while you're adding the unit tests.

Improve this answer
1
  • 1
    I would caution against automating these with a "might as well" approach. It is VERY expensive to do it right, expensive and pointless to half bake it. If you decide to automate, treat it as a project. Put togetehr a business case. Allocate time and cost to it.
    – mattnz
    Sep 13, 2012 at 22:34
10

Since the test is 10 years old and it talks about "message Z is displayed", I would call this UI Regression Testing - testing to see that recent changes haven't broken older UI functionality.

The only way to find out if the test is still useful is to do your best to test with it! Automated application-(non-user)-interface testing is always a win and Selenium is really sharp, but the effort to maintain any automated UI test is often equal to or greater than the effort required to test manually every release. The advantage of manual testing is that (in theory) you can be tolerant of visual UI changes on a screen with unchanged functionality. In fact, this is exactly what you want to test for in the user-interface and exactly what makes automated testing report false positives.

I find the hardest part of writing a UI test is getting the right amount of detail so that testers stay alert to the experience a user would be having with the system instead of focusing so much on the test-plan details that they miss the bugs. I haven't closed that question yet, but the sense I'm getting is that it may be useful to have a Selenium test in addition to a manual UI test. So that the details are tested by Selenium and the over-all experience is still tested by humans who can test it better without being bogged down by an over-detailed test plan.

Every kind of UI testing has its problems. There is no substitute for an interested person who really wants to find bugs.

Improve this answer
1
  • Nice observations, especially about false positives. User testing can be very deceptive if not properly curated and maximizing the time and focus of the test-group is vital. Offloading intensely specific testing while having real user tests does seem ideal.
    – That Realtor Programmer Guy
    Sep 17, 2012 at 4:16
6

This looks a lot like the system tests we used to use about fifteen years ago. Most of the ones we used couldn't be automated, because they had preconditions like configuring specific hardware (this was for telecom switches) or had test steps like "while data is transferring, apply a 100mV square wave signal to the TxD line and sweep the frequency from 50 to 120Hz repeatedly".

I'd recommend you automate as much as you can, but don't expect to eliminate everything on your test plan. Some degree of manual testing should be done for any interactive system.

Improve this answer
3
  • You can also get devices that will programmatically generate signals.
    – Paul Nathan
    Sep 13, 2012 at 17:04
  • @Paul - sometimes, partuclaryly with legacy systems, it's just easier and cheaper to get a person to do it.
    – mattnz
    Sep 13, 2012 at 22:39
  • @mattnz: sure. I just wanted to point out that automated testsets CAN be used and should be considered.
    – Paul Nathan
    Sep 14, 2012 at 16:16
1

You may still use that file to test and there are benefits of human testing of UI part of the application. However, you may also look at automation part where you have couple choices.

I would look at alternative file or test objects input/output testing, if it is hard (even not possible) to add unit tests to your inherited project. Basically, you need to prepare input test files/objects to feed your program and compare the output with the one that has expected results. This matching mechanism might help to discover them.

You may use the existing structure of testing to build your input/output files or objects. In addition, because this code is 1o years old, i would double check this tests with business analyst or the subject matter expert to verify the tests, and cover the all important business processes that application should provide.

Although it may look like not the best way of automation, but it is extensively used in old medical billing systems.

Improve this answer
1

This might be called an "Acceptance" test. If it dictates what the program must do to be considered "complete", then I would call it an "Acceptance" test. It depends on when this testing occurs in the development cycle as to what name it is called.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.