We are trying to setup our testing process. We wonder if our testers should develop automated regression tests, or if the developers should do that.
And what about other types of automated tests? Should testers develop them?
Software Engineering Stack Exchange is a question and answer site for professionals, academics, and students working within the systems development life cycle. It only takes a minute to sign up.Sign up to join this community
I say the testers should develop the automated tests - they have the "outside pair of eyes" approach to the code, and will (or should that be just 'may'?) spot bugs that you haven't thought of, let alone handle.
Plus their understanding of the functional requirements might be higher level than the understanding of the developers - so sitting in between the hardcore low-level knowledge of the programmer, but not as high level than that of the BA.
In my opinion, developers and testers are responsible for different types of tests.
The developer, while writing the logic, should be also writing unit and integration tests. This will allow the dev to make sure that what they've written thus far works as intended. Additionally, these tests will still be around for the dev to run when they make future changes. Once the logic is completed, the dev can be assured that what is written works as they understand the specifications and can pass off to the tester.
The tester from this point should be responsible for writing system wide tests that make sure the business logic works as intended.
Given that devs are often way too attached to the code, testers should be able to help clean up the dev's tests but not vice versa.
In my experience, the way a tester sets up and executes a test case automatically does actually differ from the way a developer does it. Testers will be thinking more about how to get the most information out of the test case, how to execute tests rapidly, how to keep tests maintainable, and so forth. Most importantly, testers will see nuances of test coverage that developers will miss.
If test development resources are low, developers can help out - but a tester should review their work carefully. Devs should work on fixtures and test tools before writing actual test cases, and devs shouldn't ever write test cases for their own code. Having developers help out with testing does have perks - it exposes devs to other pieces of the product, and testing can help them become better developers. However, just as a junior developer's work would never go without a code review, a dev's QA work should get a QA review from someone with more experience testing.
Edited to add: I'm an SDET with 5 years of experience. I work with great devs with 10+ years of experience each, and lately have been working with them to get through a testing bottleneck.
One critical distinction that's really important here is this: are your testers simply checking, or are they testing?
This blog post by Michael Bolton explains it better, but in essence: are they looking to merely confirm behaviour, or are they looking to find issues with the system?
I think it's also useful to consider the Agile Testing Quadrants(Brian Marick originally described these, but I came across them in Lisa Crispin and Janet Gregory's "Agile Testing" book: even if you're not following an Agile development methodology, I think the distinction between tests that critique the product, and tests that support the team, is really worthwhile when considering automation, and trying to develop a plan for who does what, and why.
For instance, unit checks written by developers act as change detectors, enabling you to catch regressions early when they're re-run regularly - these are tests that support the team. System level regression checks that are automated so that they can be re-run regularly and quickly also support the team by catching regressions early, and complement the unit testing done by developers. That frees your testers' time up to do testing that critiques the product - exploratory testing, for example. Or possibly applying some of the automated checks to stress test the product.
The other thing I really like about the Lisa Crispin presentation I linked is that it points out that automation can also be used to support manual testing - creating test data, automation used to get a scenario to the point you want to focus on today, for example.
Considering these two articles will hopefully help you to analyse what sort of testing you want to do, make it easier to pick out what might be suitable for automation, and figure out which bits of automation are more suited to be done by testers, and which by developers.
One thing I would really like to be able to see is solid automation tools for testers that will allow them to effectively record their progress through a test script and then permit that script to be run automatically in future. Especially if this also facilitates the same script being run on different operating system versions without the tester having to go through all of them by hand.
Unfortunately, certainly for the product that I work on, none of the tools on the market quite do the job. But it is worth bearing this in mind and looking at what is available on the market in case there is something that that would work for what you are doing.