1

At work I am currently tasked to implement End2End/integration Tests for one application using Selenium.
we have an project consisting of a frontend and multiple backends (spring-boot apis). The project is worked on and new features are getting developed and old ones adjusted. At the moment there are no tests and frequently features break in dev or test deployment. My goal is to have an extensive End2End test Suite covering most if not all features so we can assure no features gets broken with an update. Currently we test things manually costing time and resources, thats why we want to automate

I have a basic setup going, where I start all components using Testcontainers so I can run/implement my tests fairly simple.

I am struggling with creating a good Test structure. I have other projects at work for reference but it seems like every project does things differently, from selectors to page objects to abstraction, there is no consistency.

Basically I am looking for references for good Selenium tests and or personal experiences from working and integrating tests with the framework. What worked well for you or what failed short or long term. Maybe even things you did differently from the documentation or general recommendations because they worked better for you.

Looking forward to hear your experiences - thanks in advance

3
  • One thing that works well is document why the test exists. That way people know when to delete it. Without that people drag obsolete tests around forever. It might help us help you if you told us why the tests you propose creating are going to exist. Commented May 28 at 14:53
  • @candied_orange we have an project consisting of a frontend and multiple backends (spring-boot apis). The project is worked on and new features are getting developed and old ones adjusted. At the moment there are no tests and frequently features break in dev or test deployment. My goal is to have an extensive End2End test Suite covering most if not all features so we can assure no features gets broken with an update. Currently we test things manually costing time and resources, thats why we want to automate Commented May 28 at 15:44
  • Thank you. Please edit supplementary info into the question. Those attempting to answer shouldn't need to dig through the comments. Commented May 28 at 15:49

2 Answers 2

2

I have been organizing and implementing end-to-end test frameworks for more than 10 years. I have to say, I am frustrated by the lack of design patterns and architectural guidelines for this kind of testing. I feel this is a very under-developed area of software engineering. While I can't describe The One Test Architecture To Rule Them All, I can say confidently that my test code is just as purposefully designed as the application it tests.

Make It Easy On The Test Author

Start with the test. Presumably you'll end up writing dozens — scratch that: hundreds of these kinds of tests. Yeah, yeah, I know. Testing pyramid, unit tests, refactoring blah-blah-blah, but none of that is possible for an old application that wasn't originally written with unit testing in mind. If you just inherited 100,000 lines of spaghetti code, you are likely only able to test this thing from the user interface. Now software architecture for your tests become relevant.

So, start by writing an end-to-end test from the perspective of the Test Author. Make this easy on the Test Author. This poor person will be writing these until their fingers bleed. Think in terms of the Arrange, Act, Assert methodology for writing a test. What setup do you need? What is the System Under Test? How do I interact with the System Under Test? How do I make an assertion?

Some guidelines for making tests easier to write:

  • Provide a test framework that allows the test writer to specify as little information as necessary. If the name of a user is not relevant, don't force the Test Author to name the user.
  • Provide a clear distinction in code between the Arrange, Act, and Assert phases of a test.
  • When you need to name things or specify information in a test, prefer strings or other common types that are easy to hard-code.
  • Each step or part of a test should ideally be a one-liner.
  • Settle on a naming convention for your tests.

Consider a use case where you need to write a blog post. The use case needs a registered user who has already created a blog, and now needs to create a blog post. Please forgive the C# syntax, as this is what I'm most familiar with.

public void CreateYourFirstBlog()
{
    // Arrange
    Given.User.Register();
    Given.User.Login();
    Given.User.CreateBlog();

    // Act
    var viewPostPage = When.Blog.CreatePost(title: "Easy on the Author",
                                            body: "<p>Make it easy on me, please!</p>");

    // Assert
    viewPostPage.PostTitle.ShouldBe("Easy on the Author");
    viewPostPage.PostBody.ShouldBe("<p>Make it easy on me, please!</p>");
}

The "Arrange" parts are prefixed with "Given", not because this is some standard, but because it matches the Given-When-Then flow of a behavior-driven test, which is just a personal preference. These steps of the test can insert data into a database, or call web services to set up data. They can also interact with the real system under test via Page Models, too.

The "Act" part is prefixed with "When". This part should mostly interact with the System Under Test via Page Models.

Sometimes I have a "Then" property for assertions, but sometimes I go straight to the page models for the assertions. I don't have any real guidance here. I've used both ways, and either one works best in different scenarios.

The point is, the test should be dead easy to write. Only name things relevant to the part of the system being tested. Your test framework can auto generate stuff that isn't relevant. For instance, the name of the user doesn't matter, so you could call them "user2134" or "author{timestamp}" or whatever. The test framework is responsible for juggling this information, not the Test Author.

I haven't found a good design pattern for these quick one-liners in the test. The closest I've seen is the Test Driver Pattern, although the example I link to is not necessarily the best example. I also might be thinking of the Screenplay Pattern, which uses page objects under the hood. I didn't really think about it until writing this answer, to be honest.

The less you need to specify in your test, the less likely things will break when the user interface changes.

Separate Your Testing Concerns

This next bit is straight-up software architecture and design. Most people dismiss this part, because tests are the last thing we think about. It's just test code, right? All we need is that green check mark in the test runner. Just slap this baby together and ship it out the door as soon as we get that green check mark. Crying and gnashing of teeth quickly follow. This is the spiral to Testing Hell, because you've created a brittle test that fails every time something changes.

Over the years, I've discovered the following things are usually separate, and deserving of their own classes, and sometimes their own libraries:

  • Test startup and configuration. Think: Composition Root for your tests. This big tree of objects needs to start somewhere.

    • Complimentary concept: dependency injection. A DI framework is not the worse thing to introduce, and having a clearly defined Composition Root for your tests gives you a place to wire up all of these objects and dependencies. It can be as simple as a "before test" method in some common base class for all your tests.
  • Test teardown. Find one spot in your code that should run at the end of each test to do any cleanup work, be it data, reporting, or taking screenshots. Again, most test frameworks provide a "teardown" or "after test" method. Push this into a common base class for your tests.

    Which leads me to...

  • Create A Base Class For Your Tests. This gives you one spot to accomplish the previous two points.

  • Use Page Models. But not everywhere. And be true to the pattern. When a method on a page model causes a transition from one page or screen to another, that method should return a page model representing the part of the application you expect to be taken to next.

    • Provide "happy" and "unhappy" paths as separate methods in your page models. The LogInSuccessfully() method should return a HomePage object, whereas the LogInUnsuccessfully() method should return a LoginForm object, because we expect the user to remain on the login form.

    • Data Hiding Is Your Friend with page models. Do not expose page elements. Expose methods that give you a suitable abstraction to work with, so the test author doesn't need to induce WebDriverWait. The page model should handle that nastiness.

    • As a compliment, encapsulation is your friend with page models, too. The basics of object-oriented design are equally important here. Remember that the user interface is going to change. You don't want to change 1,000 method calls just because someone started fading a menu in instead of just setting display: block in CSS.

      Come to think of it, this is precisely what the Single Responsibility Principle is talking about. If something breaks, you should only need to fix it in one spot.

  • World Objects are a great way to present the outside world to the test framework. This class, which I usually name after the application, is responsible for presenting the outside world to the test framework. Typically page navigation begins here. This is also where some initialization work can happen when using something like Appium or Selenium. This pairs well with the Composition Root mentioned earlier.

  • Use proper Javadoc comments on all of your test infrastructure code as a way to help guide Test Authors. Or JS Doc. Or whatever doc you use for your tech stack. Writing tests is a complex endeavor, so auto-complete assistance in an IDE can be crucial.

  • A Test Context is a great way to stash information used between steps in your test. This also gives you easy access to information created during the setup of a test so you can clean it up at the end. That "unnamed" user you created using Given.User.Register()? Stash the username or user Id in the Test Context so page models, or other test infrastructure code can refer to it later. Pass this text context around via dependency injection to any other object that needs it. This also pairs well with the Composition Root and a DI framework.

  • Utilize something like the Test Driver or Screenplay patterns. It takes some work to set up, but this extra time gets paid back faster with the more tests you write. And don't forget that you need to maintain these tests. A cleaner looking test won't need to change as often. Things like page models will see a lot more changes.

  • Keep the automation framework separate from the tests. This allows you to split up large test suites along arbitrary lines. You could split things along modules, features, kinds of users — doesn't matter. The tests use the framework, but the framework can exist on its own. When your test suite gets messy, split it up. Keeping your automation framework separate allows you do reorganize your tests.

Do not use Gherkin for QA testing!

I'm going to get death threats for this one, but Cucumber and similar BDD frameworks are meant to describe business processes, not edge cases. Sometimes you just need to procedurally do 15 things and double-click a button. That isn't a business use case. That's a QA test. It might bring value, but nobody outside of the engineers will care. Automation Panda has a lot of information about behavior-driven development, including a good refutation of the heading above this paragraph.

The philosophical overhead of BDD gets in the way. The overhead of proper grammar and plain language communication suddenly becomes relevant. While the aforementioned refutation for using BDD for QA testing brings up some good points, I've found code-only tests to be much easier to work with when testing fiddly use cases.

I got a test working once to verify the collapse and expand behavior a giant tree of checkboxes for all of the North American Industrial Classifications using BDD. It was horrendous to look at, and executed slower because I couldn't control when the browser opened or closed. It was much cleaner as a C#-only test where I could control when the browser opened. Turns out reopening the browser window on every test permutation added 10 minutes to the test execution time, but using only one browser session made for a flaky test. I settled on about 50 test permutations per browser window before closing it and reopening it. That gave a very stable test that only had to open a few browser windows.

Summary

This barely scratches the surface of test architecture. The bottom line is that your test code deserves the same thoughtful treatment that application code deserves. Get to know the tenets of good object-oriented design. Test automation has its own set of problems which require unique solutions. You aren't a QA tester. You are a Test Automation Engineer, with an emphasis on "engineer." Good programming practices are just as crucial in tests, so don't be afraid to put the time in to create a robust test framework that can evolve with the application it tests.

0

every project does things differently, from selectors to page objects

Initially Selenium IDE will help you identify those, similar to using Firefox CMD-I inspector to click on part of the page and see the relevant selector. Click on Export -> JavaJUnit to see what code would reproduce the user flow that your newly developed feature needs. Either use that as a starting point which you edit, or refer to it as a cheat sheet while developing a new test.

Remember to WaitForElementVisible rather than just blindly waiting an arbitrary number of seconds prior to clicking the element.

Longer term, build consensus around a standards document that addresses the "does things differently" concern. Convince folks that "testing" is part of "the release process" which everyone is trying to support. Devote more testing resources to projects that try to adhere to the standard.

Not the answer you're looking for? Browse other questions tagged or ask your own question.