5

A website I'm creating is going to have three breakpoints: desktop, tablet landscape, mobile. Should I write a test for each resolution?

  • 3
    In the name of maintainability and KISS, you'd probably not want multiple tests, unless that ends up being impossible due to the way you set up your DOM and responsiveness. – BrandonV Jun 19 '14 at 20:52
  • If your layout changes drastically between e.g. desktop and mobile, maybe separate tests could additionally check the parts that actually change on mobile, e.g. key controls are visible / clickable. Likely it will be a relatively small additional part. – 9000 Feb 13 '18 at 20:15
4

It is unlikely that you catch layout-related regressions with Selenium tests:

  • The regressions related to how elements are displayed lead to layouts which look broken, but still have all the elements needed to perform the test. If you ask Selenium to enter text in a field, click on a button and wait for specific text to appear in a specific <div>, Selenium won't be able to know that the button is too short for its contents or that the text input has a wrong position on the page, or that the text within the <div> fills two lines instead of one.

  • The regressions which move the elements out of the drawing area (which leads to test failures, if, for instance, Selenium is asked to click on a button which is not visible) are too random anyway and usually lead to flaky tests.

You can, however, use pdiff tests to detect changes in layout: the goal of those tests is specifically to catch visual regressions. If you target three screen sizes, it is indeed a good idea to run the tests for all three sizes. It is not unusual to use four or five sizes, and is especially valuable given that developers rarely check how their changes influence the layout for very small or very large screens.

1

Use a design spec test to validate computed CSS properties for each breakpoint. For example:

responsive test spec

At the top of the spec file, we define named elements using CSS selectors. These can then be used throughout to refer to elements by name.

We then group element checks by viewport size so that we can make different checks at different responsive breakpoints. Named viewport sizes like small, medium and large are entirely customisable. The exact viewport dimensions are specified in a global test suite file that isn’t shown here. The * group runs checks for all viewport sizes.

Within each viewport group, we then provide the name of the element we want to check. Within each of these are the actual checks we want to carry out. Checks tend to focus on dimensions or relative position to other elements in a component. It is also possible to directly check final computed CSS properties, like text color or font size.

References

  • I absolutely don't understand your answer, and see no relation with the question. I won't downvote it since I also answered, but you may check it twice to see if you can reformulate it so the relation with the question could be more obvious. – Arseni Mourzenko Feb 13 '18 at 19:50
  • @ArseniMourzenko: In summary: Paul is proposing that OP creates a design spec in Galen Format (Galen is the final link in the answer), then use Galen Framework to test the spec. This approach requires more up front work, but it provides a closer link between a specification document and an automated test. Teams with a highly formalized design process will probably find Galen's approach preferable to pdiff. However, teams which allow QA to make frequent, un-vetted design changes will find that maintaining the spec document introduces substantial extra work. – Brian Feb 16 '18 at 21:06
  • I'll note that Galen framework also includes image comparison support, probably similar to pdiff. – Brian Feb 16 '18 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.