We recently had lots of problems with automated tests in our team. Part of it was that the people designated to writing them had little experience. After this failed, we incorporated a different approach where the product owner and the developers are much more engaged in the overall process of automated testing, so the quality improved.
There is still one more problem that we are facing without a good solution and that is the test data and it's maintenance.
We have integration tests written using RestSharp and xUnit (the back end is ASP.Net Core) and GUI tests written with the use of Selenium and xUnit (the front end being in Angular).
Both of these rely on MS SQL as the data store.
Now best practices state that the tests should be as independent as possible from each other. This is quite easy to accomplish in unit tests where we can arrange test data because the tested part is very small. In GUI/Integration tests however, matters are a bit different.
Since GUI/integration tests often perform complicated actions (which test many more elements than unit tests), it is very difficult to create independent mock data in a database for each individual test.
Let's assume that we want to write automated tests for some hierarchical organization structure which is added through CRUD screens. We have sites with child areas which then have child subareas and so on.
The first test would be to test adding and editing Sites. This is quite simple. The second test for Areas requires assigning a preexisting Site in the system so we can create hierarchy. Third would require Site and Area and so on.
So basically the more complex the test scenario, the more preexisting specific data it may require and failures at the beginning of the hierarchy-add will fail other tests which may work fine on their own.
Tests rely on each other for creation of data - This is in my opinion a very bad approach because failing the first test will fail the rest. Plus it doesn't scale very well for more complex scenarios.
Each test loads it's data individually - this allows the tests to be independent but because of this we will have lots of code just to load the data for the test in case of changes or problems it will also cause a lot of rework so it will also make the tests a bit fragile.
We could use containerization with Docker. Create database image and load the database with data on each run. This will allow us to make sure that more tests can be run in parallel but again solution 1 and 2 would also need to be implemented to push the data into the container.
We could divide the tests into different categories. Simple/Complex ones. Simple one we would assume can be written in as much isolation as possible. Complex ones could have shared steps (Hierarchy would be such an example and we could accept that there will be some chained test steps there).
Any guidance here would be greatly appreciated. Creating successful test automation is a lot of work and the best thing is to start in the best direction possible.