3

I have a very large selenium test framework I use to test a web application. It it built around a page-object pattern.

This week I fixed a bug in a low level piece of the framework, but it broke many other pieces that got built on top of this bug. I didn't have a good way to find these spots without running the full test suite of several thousand UI tests.

Since this isn't ideal, what strategy can I use to unit test new functionality and page-object modifications/additions to the framework?

  • Can't think of anything but brute force. Do you have a build system that can distributed the work between several agents and run the tests in parallel? – M.P. Korstanje Jan 16 '16 at 0:30
  • @mpkorstanje The test infrastructure is designed so that all tests are run as parallel as possible across as many browsers as we support. However, I had to run these test suites multiple times as I fixed each issue. – adam Jan 16 '16 at 0:33
  • It should be noted that I have the concepts of Mock objects, serving my own contrived pages on localhost when running the unit tests, and a couple more basic unit test strategies running around in my head. I am having a hard time connecting them to a coherent test plan. – adam Jan 16 '16 at 0:35
4

When you change the behaviour of a component in your framework, then just unit-testing that component is not sufficient. You must also re-run the integration- and higher-level tests that involve that component to verify what effect your changes have on the rest of the system.

If you find that the effect of your change is very large, the either you should reconsider the need for the change or bite the bullet and iteratively address the adverse effects.

2

Quis custodiet ipsos custodes?

Your test framework had bugs in it, which means that your tests were buggy. This is the problem with complex tests and complex testing systems. One way to deal with this is to create unit tests for the complex parts, and then confirm that they behave correctly.

In other words, it's just software, and if it isn't dead simple, it probably needs testing.

1

You will need to run all tests eventually as you're making a fundamental change. However you don't have to run all tests at once. Rather then fixing the bug, you could deprecate the bug causing functionality, create a new and correct bit of functionality which over time replaces the old functionality.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.