We use a detailed step-by-step user-interface regression test for our commercial web application. It has a "backbone" test for the most used / most important parts of the system, with optional tests for specific areas of functionality. Using this plan has definitely helped us ensure high quality software.

But, having very specific tests can be counter-productive. The tester concentrates on following the test and will completely miss usability issues, or not notice fairly obvious problems such as the bottom part of a page that is missing.

By contrast, some of the best UI testing happens when building a demo of a new feature. I often do my own best testing by pretending to demonstrate the system to an imaginary prospect. Yet when I tell the testers, "Just demonstrate the system to yourself" they don't cover nearly as much functionality as they do with a detailed point-by-point test.

I'm repeatedly asked to provide more and more detail in the test plan so that a new untrained tester can test with it without asking any questions. Yet details seem to be counter-productive. How much detail do you put in a regression test to make it effective? How do you write tests that make the tester focus more on the system than on checking off items on the test?

  • I have a feeling that you know the answer better than anyone else
    – superM
    Commented Aug 31, 2012 at 13:33
  • What answer is that? Test it myself? I'm posting here in hopes of finding a different answer! :-) Commented Aug 31, 2012 at 15:41
  • 2
    First, its not an answer but a comment. Second, no one here can know your team better than you do. So, since you tried many things and were looking for even more, I assumed you'll be able to know what's best for your team!!!
    – superM
    Commented Aug 31, 2012 at 19:15
  • Should I have posted this on the User Interface forum instead? Commented Sep 13, 2012 at 14:16

3 Answers 3


Technically speaking, a regression test should not be looking for usability issues. It may catch deviations from the UI design as part of the test cases but usability issues are best determined by UI design specialists. UI regression tests are usually performed by QA (testers), as you know.

If the product is too big for a detailed UI design, the UI designer produce should produce guidelines that the development team should follow and the QA should check for. For example, "Each page should contain a header with A, B and C" or "When asking for an address, it should have these fields laid out in this way".

It may be better to supply the UI verification steps in a different way. For example, instead of giving steps, include screen shots of that the app should look like (which should catch the bottom portion of a page missing).

You may also want to consider an automated UI testing tool like Selenium. This can automate much of the repetitive testing aspect you are capturing in the steps and free you and your testers up to look for harder to reproduce or more critical issues.

  • Ooh - Selenium looks pretty cool! To look for usability issues, do you do Usability Testing, Beta Testing, or what? Sometimes we change something and it makes a distant part of the system awkward/misleading/un-intuitive. Commented Aug 31, 2012 at 15:37
  • In my organization, we have a specialist UI design team that development teams can use if needed. The UI design team, product managers and architects iterate until we have a good UI design, which is then implemented. If there are questions or contradictions, the product managers and architects can choose to involve the UI design team again to fix them. This may or may not work in your case.
    – akton
    Commented Aug 31, 2012 at 15:56

You cannot command someone to be innovative. If you have only testers in your team without developing own ideas for good tests, I don't think a better test plan will solve that problem for you. Either they need more experience and training, or you need different people in your team.

Said that, I think you should have both:

  1. a very detailed step-by-step check list of things where you want testers to have an eye on

  2. some general questions / guidelines, for example, about testing usability or about things not explicitly mentioned in your plan, or a demand for creating missing points in the checklist by themselves.

Make clear that you expect your team not only to focus on point 1, but also on point 2, but don't expect too much because of what I wrote at the beginning.


If you say "demonstrate the system", I hear customer workflows: how the customer is going to use the system.

So typically, next to some basic functionality, like "does it install, start etc.", I find UI testing mostly helpful to simulate a real user working with the system.

Also, most often, UI automaters get their specification from some other place. If there is no such place/person, you could ask them to talk to customers or product management on how they use the system.

This also has the benefit of you being a little more certain that the product you ship is working in (at least) some real world scenarios.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.