Writing Selenium Tests - Modifying Markup Under Test [closed]

Question

We'd like to know what you view as best practices and what has worked for your projects

My organization is in the beginning phases of creating a Selenium test suite, and have had some early discussions about how to ensure that our tests are robust.

In one of the first tests, we modified the markup under test to facilitate testing:

The original markup was something along the lines of:

<div><a href="someLink"><i class="fa fa-someicon"></i>Some Text</a></div>

And the author of the test added an ID to the anchor tag:

<div><a id="SomeId" href="someLink"><i class="fa fa-someicon"></i>Some Text</a></div>

A few points of view emerged:

1. Adding anything to the markup solely for the purpose of facilitating testing is a bad thing.

   a.) It clutters the markup, which hurts front-end developers (i.e. where is this id even being used?).

   b.) It increases the payload being sent to the clients.

2. Adding an id to an element to facilitate testing is okay.

   a.) It allows for more robust selectors (traversing the DOM / using a XPath can be much more brittle and difficult to read).

   b.) Assuming that you use unique id's, there's no material downside to adding identifiers to elements on a page.

3. Adding another attribute (i.e. data-testing-id) is preferable to adding an id.

   a.) While it clutters the markup and is also sent in the response to clients, the purpose of the attribute is very clear.

   b.) Similar to point 2, you can write more robust selectors.

Answer 1

Adding anything to the markup solely for the purpose of facilitating testing is a bad thing.

Not necessarily. Dependency Injection means injecting dependent classes into other classes, instead of instantiating them. One of the main reasons you do this is to facilitate testing, which has value in and of itself. Every system, from relatively simple cars to ships, airplanes and spaceships has things like maintenance hatches that are not meant to be used in the normal operation of said system, but to facilitate maintenance. Imagine if cars didn't have a hood...

Adding an id to an element to facilitate testing is okay. Adding another attribute (i.e. data-testing-id) is preferable to adding an id.

There are probably alternatives to adding an id in some cases. You could simply look for a link that displays a certain text if you wanted to. If your tests are well-designed, even a change to that text would only result in a very limited change to your tests.

Remember that, at this level, your test is interested in how your user interacts with the application. A user does not 'click the button with the id buttonSave' but he 'clicks the button labeled save'. If you suddenly change the labeling of the button, you will not only break the test but also break what the user is used to seeing. If you have multiple buttons labeled as 'save' all of a sudden and you break the tests, you might have a problem with your interface. Both things are worth being notified of by your tests.

Answer 2

I've worked with small and medium-large Selenium-based test suites. We've had discussions like this with all the same points of view. In my opinion, the best approach is this:

• add id attributes where possible
• if it's not possible use other attributes like class for use with automation
• only add tags as needed/requested for testing. Start bottom up instead of top down.

Adding id attributes can make front-ends easier to test, easier to automate, and more organized. These are generally Good Things.

I'm a bit troubled by this statement though:

Adding anything to the markup solely for the purpose of facilitating testing is a bad thing.

Here's the problem with this statement: you're focusing on the wrong problem. Instead of asking "how can this cause problems with our app", ask "how can we make our app more testable?"

It's a good practice for elements in the DOM to be designed so that they are uniquely identifiable. That's part of the point of the id attribute. Some elements naturally lend themselves to having an id like an OK, Save or Cancel buttons. Sometimes elements in a list or grid can't realistically be given an id which brings in using other attributes or some other approach.

Also, the idea that using id can hurt the payload requires evidence. Usually that's just premature optimization-style thinking. In our app, adding ids slowed down page loads by less than 100 milliseconds on average, a negligible amount in our context. Perhaps adding hundreds of id attributes might hurt performance on some platforms, on some browsers but again, investigate. We're not living in 1996 on the web anymore.