Company outsourced testing; how can we encourage programmers to stop overly relying on testers?

Question

A friend of mine is working in a 200-employee company. The company's business has nothing to do with IT, but they do have an IT department to work, among others, on their website, used by the customers.

The website started with a core idea that programmers have to test the application themselves using automated testing. However, it quickly started to be problematic, as programmers were spending too much time writing functional tests with Selenium (and later Cypress.io) trying to deal with either complicated interactions, such as drag and drop or file uploads, or trying to figure out why the tests randomly fail. For a while, more than 25% of the time was spent on those tests; moreover, most programmers were pissed off by those tests, as they wanted to produce actual value, not try to figure out why the tests would randomly fail.

Two years ago, it was decided to pay a company from Bulgaria to do the functional, interface-level tests manually. Things went well, as such testing was pretty inexpensive. Overall, programmers were delivering features faster, with fewer regressions, and everyone was happy.

However, over time, programmers started to be overconfident. They would write fewer integration or even unit tests, and would sometimes mark features as done without even actually checked if they work in a browser: since testers will catch the mistakes, why bother? This creates two problems: (1) it takes more time to solve the issues when they are discovered by testers a few days ago (compared to when they are discovered within minutes by programmers themselves) and (2) the overall cost of the outsourced testers grows constantly.

Recently, the team lead tries to change this behavior by:

• Measuring, per person, how many tickets are reopened by the testers (and sharing the results to the whole team).

• Giving congratulation to the persons who performed the best, i.e. those who have the least tickets being reopened.

• Spend time pair programming with those who performed the worst, trying to understand why are they so reluctant to test their code, and showing them that it's not that difficult.

• Explaining that it's much faster to solve a problem right now, than to wait for several days until the feature gets tested.

• Explaining that testers do system tests only, and the lack of unit tests make it difficult to pinpoint the exact location of the problem.

However, it doesn't work:

• The metrics are not always relevant. One may work on an unclear or complex ticket which gets reopened several times by the testers because of the edge cases, and a colleague may meanwhile work on a ticket which is so straightforward that there is absolutely no chance to introduce any regression.

• Programmers are reluctant to test code, because (1) they find it just boring, and because (2) if they don't test code, it looks like they deliver the feature faster.

• They also don't see why fixing a problem days after developing a feature would be a problem. They understand the theory, but they don't feel it in practice. Also, they believe that even if it would take a bit longer, it's still cheaper for the company to pay inexpensive outsourced testers rather than spend programmers' time on tests. Telling them repeatedly that this is not the case has no effect.

• As for system vs. unit testing, programmers reply that they don't spend that much time finding the exact location of a problem reported by a tester anyway (which seems to be actually true).

What else can be done to encourage programmers to stop overly rely on testers?

Answer 1

It seems to me there is a contradiction in policy here.

On the one hand, the firm has outsourced testing because it consumed programmers' time excessively, and could be done more cheaply by others.

Now, they complain that the programmers are relying on the testers, and should be doing more testing themselves up front.

I can understand from a management point of view that there is perceived to be a happy medium, but in reality the programmers are not engaging in a close analysis, on a case-by-case basis, of how much testing they do themselves and how much they outsource.

To attempt to do so would consume too much time and intellectual effort, and likely without producing accurate results. How would a programmer go about estimating how many bugs a particular piece of code has, and then weighing up the economic benefit of spending his own time searching for them versus letting the testers search for them? It's an absurdity.

Instead programmers are following rules of thumb. Previously the rule was to test extensively. Now the rule is to save precious programmer time, get more code out the door, and leave testing to testers (who are thought to be ten-a-penny).

It's no answer to seek a happy medium, because in practice what will happen is that the anal-retentives will return to spending 25% of their time testing, and the cowboys will continue throwing low-quality code out the door, and personality traits like conscientiousness and attention to detail (or lack thereof) will predominate over the judgment. If management try to harass both types to get them to conform more closely to an average which is perceived to be economically ideal, both will probably just end up feeling harassed.

I would also remark in passing, that the 25% of time which was spent testing to begin with, does not strike me as excessive.

Answer 2

Bottom line: this is a cultural problem.

I come from the viewpoint that competent programmers at least write unit tests for the more complex parts of their code. The problem is not everyone shares my viewpoint. I've known people who have been developing code longer than I've been alive and they also test their code — just not necessarily with automated testing. There are a number of things in software development that are too simple to test so those tests hold no real value.

That said, there are different levels of testing with different percentages of random failures that can happen. From a management perspective, you have to understand where you gain value.

• Unit tests: check implementation details and error handling as close to the logic as possible.
• Integration tests: check that system specifications are working correctly
• User Interface tests: check that the application behaves according to the requirements

Unsurprisingly, the further you get away from the individual code units, the more brittle your tests become. The more pieces you have that must work together, the more chances that something will intermittently go wrong. What that means is that the closer to the unit test you can catch problems the more reliable and valuable those tests are.

Cost of Automation

Automation costs time. Time costs money. It takes longer to write automated tests than it does to manually test a feature. However, each time that automated test is run, it runs in a fraction of the time of manual testing. From a management standpoint, you want to make sure you have a good return on your investment. I highly recommend that the highest risk code — if it breaks the application is useless — should have automated tests to ensure that you catch regressions (broken code) as soon as they occur. If the unit tests don't pass, the developer can't push their code.

In general it does help to have some guidelines, and a means of ensuring high risk code is covered.

• Unit tests should have at least 25% coverage. (I personally prefer higher, but for a team with no unit tests this is a good place to start)
• Unit and integration tests should be prioritized on high risk code first.
• Definition of Done needs to have one or both requirements:
  • Code peer review (Pull requests are a great way to organize these)
  • Unit test coverage meets minimum criteria

Cost of Manual Testing

Manual testing costs time. Time costs money. While it is faster to manually test a feature one time, it takes the same amount of time to test the feature each time. You want to keep testing finished features to protect from regressions. Regressions are functionality in your application that used to work that don't any more.

The hidden cost of manual testing is that testers are people, and sometimes people skip tests on accident. If you thought writing automated tests was tedious, try testing the same features with all the button clicks time after time.

Optimizing your investment

Here's where both management and development have to be on the same page. If quality is important to the company, then the company has to be willing to invest in quality. If quality is not important, then just do away with testing. Your users will complain, and you may or may not be embarrassed by the problems they complain about. That said, if the application is mission critical, then quality should be important.

• Automate testing high risk code
  • Risk can be high due to the complexity of the solution
  • Risk can be high due to how necessary the feature is
  • Risk can be high due to the high number of dependencies on the code
• Don't write tests for code that is too simple to fail (like getters and setters)
• Manually test things that are too complicated to test automatically (like drag/drop)
• Invest in simplicity
  • A requirement on its own might be simple enough, but may conflict with other requirements.
  • Be ready to remove features so the application serves current needs.
• Define "done" so it is clear and unambiguous
  • When the work that needs to be done is unclear, then developers and testers have different opinions on what's right.
  • A few more minutes in a meeting with three people can save days of work and rework because of differing definitions of done.

Summary

The company culture is currently in a no-win situation. Culture changes are easier when management has buy-in. It's also easier when the team introduces disciplines that help them be more effective. To that end, I would prioritize Defining Done before prioritizing anything to do with how tests are performed.

It's great that you are collecting metrics. It's not great how those metrics are currently being used. A better way is to look at the trends on the metrics as you introduce more structure in how your team develops software. For example, if time to completion is improved and number of test failures are down because you spend more time defining what needs to be done, then that is a win. If you increase your automated test coverage to 50% and don't see any improvement in the number of test failures then maybe 25% is good enough.

Software development is a team activity. The more you work together as a team, the better everyone's attitude will become. The more your team is set up for success, the more your team will experience success.

Answer 3

Okay, we all share the common goal of protecting and defending developers but there are some things in your question that make me somewhat uncomfortable...

They also don't see why fixing a problem days after developing a feature would be a problem. They understand the theory, but they don't feel it in practice.

I am sorry to break the news, but experienced programmers do feel the value of catching a bug early in practice.

As for system vs. unit testing, programmers reply that they don't spend that much time finding the exact location of a problem reported by a tester anyway (which seems to be actually true).

This is textbook defensive talk (smells somewhat passive-aggressive). And of course it's true....until you put it in perspective. You can ask the programmers... do they use bookmarks in their browsers? Do they home-page a search engine? Do they use alt+tab to switch between applications/windows? All these things are pitiful excuses of productivity gains... that is if you do them ONCE. When you do it all the time, those few seconds gained easily sum up to countless man-days of productivity. It takes your programmers what, 1 minute to find the source of a problem reported by the tester? That's the time they felt. The actual time includes the time of the tester, the time to prepare the report/file the ticket, the time potentially spent in communicating the details and the time to close the ticket/report it as solved. And time is money, right?

Think of this like forgetting to buy milk when you go buy food. While you're IN the store, it's 1 minute of walking. When you are at home, it's exponentially more. Now, to put it in perspective, this is how 1 minute of felt time translates to ~30 minutes of actual trouble for all involved (though it is probably far more in reality). If management knew that proper testing could save the company some 30x the time occasionally spent given the current situation, how would they feel?

Furthermore...

For a while, more than 25% of the time was spent on those tests; moreover, most programmers were pissed off by those tests, as they wanted to produce actual value, not try to figure out why the test would randomly fail.

Again, I am sorry to break the news, but experienced programmers understand that there is actual value in tests plus 25% of the time is actually not that bad a percentage. Good tests are worth their length in gold! Also, tests don't randomly fail, at least not as often as this bold statement would have you think. Not good tests, at least, so the quality of the tests is also one thing that will have to be seriously considered. Think about the CE mark, for example, which is all about testing! Do the programmers equally disagree that it does add actual value to products?

The actual trouble with this last quoted OP statement, however, is that it seems to hint that management has actually taken advice from the programmers to outsource testing, or at least been affected by the programmers' aversion to testing, unlike how other answers may suggest that management screwed this up all by itself.

Programmers are reluctant to test code, because (1) they find it just boring, and because (2) if they don't test code, it looks like they deliver the feature faster.

Well, management does screw up occasionally, right? It seems that pressure has been built on delivering features quickly, and this has pervaded the culture a bit. Cutting corners and dropping tests go hand-in-hand when under excessive pressure to deliver, unfortunately.

Experienced programmers would go a long way over how tests are worth the time, rather than complaining that they eat up their time and they don't get to deliver value, as they say. I am in the devil's advocate position of having to give the simple explanation that the programmers are relatively inexperienced, but also, the entire system they are juggling with is not that complex, so they have not had too much trouble yet, to really feel the value of proper automated tests.

There is only one suggestion here I would be in position to give and this is to strive to make the developers feel the actual value of tests at all costs. Also, while at it, keep the external testers and compose proper automated tests to the extent possible. If the output is not too complicated, safety-critical, etc... the company could probably do without the external testers, but that always depends on the details.

Answer 4

Don't just assume you're right about over reliance on integration testing. You're pushing for the wrong things. Be very careful about driving people to measurable results vs actual results.

Measuring, per person, how many tickets are reopened by the testers (and sharing the results to the whole team).

I’m sorry but this is just dumb. The programmers job isn't done when code is in the hands of a tester. Counting how many times code goes back and forth between them against the programer is counter productive. You want programmers engaged with testers. Don't punish them for doing it. Good programmers cheer on their testers. They don't hide from them.

Giving congratulation to the persons who performed the best, i.e. those who have the least tickets being reopened.

This is not a good measurement of best. You're rewarding those who can hide their bugs, not those who work to see them eliminated.

Spend time pair programming with those who performed the worst, trying to understand why are they so reluctant to test their code, and showing them that it's not that difficult.

Everyone who runs their code is testing their code. When people don't understand how they're being judged the smart ones don't even try to guess. They just quickly submit to the judgement and hope to learn something useful from the result. We have a fancy term for this behavior. It's called agile.

Now please don't take that to mean I don't like pair programming. I love it. I've learned more sitting with fellow coders banging away on a problem then from any book or blog. Can't wait for this covid thing to go away so I can get back it doing it.

But it's not pair programming if all you do is preach. If you do this, shut up and listen to your partner. You may learn something and find your problem isn't what you think it is.

Explaining that it's much faster to solve a problem right now, than to wait for several days until the feature gets tested.

Well here's your problem. It shouldn't take several days to test a feature. Think of code like cement. The longer it sits, the harder it is to move. Get me tester feedback faster. Preferably before I start thinking about other features.

The longer it takes to catch a bug the more expensive it is to fix.

Explaining that testers do system tests only, and the lack of unit tests make it difficult to pinpoint the exact location of the problem.

Stop explaining and do peer unit testing. It shows them what you want. Programmers communicate best with code. This lets them do that.

Works like this: I write a unit test. I write production code that passes my unit test. You peer review the code, notice something hinky, and you write a unit test that proves hinkyness. You send your unit test to me, I look it over and either make it pass or we talk about what the real requirement should be. Do peer unit testing and your weak unit testers will learn from your stronger ones. Strangely enough, the strong ones will also learn from the weak ones.

Oh and you can do this when you pair program. Makes the whole thing go fast.

Speaking of requirements, if the integration tester isn't part of the discussion about the requirements, and which ones are testable, before this coding started you're out of your mind. They make the call on which requirements are integration testable. Coders should know how this shook out before they even start on a design.

You send code to integration testers after it's been through peer review & peer testing. Now two coders have picked this over early and both of them should be rooting for the testers to find any remaining bugs. They shouldn't be trying to hide them. They should be openly reporting anything that might help the integration tester find them.

It's not programmers vs testers. It's all of us against the bugs.

So stop rewarding people for creating untestable code.

Answer 5

I'm firmly in the 100% automated coverage camp. I think it actually helps me go faster in the long term, and I hate having things kicked back to me from a tester.

That being said, if people hate doing automated tests, you need to provide an incentive to not have to write them. If they think they can still maintain quality without automated tests, let them prove it. I would require automated tests for any bugs that are kicked back to them from testers, and make them optional otherwise. In other words, they need to write a test that fails if the reported bug is present, then show that it passes with their fix, as a prerequisite to getting it retested by the testers.

What I hope would happen, is the sloppiest programmers will end up having to write the most tests, which should encourage them to take more care. Code that is the easiest to break will end up with the best coverage. People will likely start preemptively writing tests for things that are difficult to test manually, and doing more thorough manual tests for tests that are more difficult to automate. Hopefully there will also be some investment in making tests easier to write and run. Very little will change for programmers who are already careful.

Answer 6

A couple of points I'm not really seeing in the other answers.

The skill sets for development and testing may overlap but some people make great testers, and sometimes they're not the same people that make great developers. As a developer, a good QA department is worth its weight in gold; forcing me to do all the tests myself, especially in an environment where someone else's code might change the behaviour of a web page between when I test it and when the combined set of changes goes live is ... the opposite. So, if you have a constant stream of changes, testing the finished result after it's been merged is quite a bit more important.

A developer will likely be able to tell you what needs testing, what could go wrong, and so on. Listen to them.

Another point I'm seeing in the OP is that at some point, after your developers had got used to using one testing framework, there was a decision to switch to another, which boasts on its website of reducing the volume of integration tests, and so on.

You didn't mention the reasons this decision was made, nor whether any training or learning time was set aside for this change. Were they expected to learn this new framework on top of continuing to work at the expected pace on feature development, bug fixing, and so on?

Answer 7

Consider taking this up in the Project Management thread.

But also – simply tell them that their work will not be accepted unless it is accompanied by automated tests. "Yes, this is part of your job." The external testers are there only to keep you honest.

As a lifetime software developer myself (as well as a consulting project manager), I can directly speak to the number of times when I feel that "my a*s was saved" by the tests that I wrote, even of my own code that no one else would work on, as I was building it. I frequently "failed to get it right." And I was sometimes astonished when an earlier test suddenly started failing. I found-and-fixed one "gotcha!" after another ... before it had a chance. And in this way I built layer upon layer of code knowing that it was correct.

Get management support behind you that "it's not accepted until it's tested, and we reserve the right to reject your tests."

Answer 8

You can encourage developers not to "overly rely" on testers by focusing on the results you're really trying to achieve (presumably features that balance the cost of development with the value the company gets from them being deployed at a certain quality level) and letting the developers decide how much to rely on the testers in order to achieve this goal. In other words, unless you're part of the development team, you're trying to manage something that the development team should be managing.

Fixing "Done"

The first thing you need to do is fix your definition of "done." You say that developers,

...would sometimes mark features as done without even actually [checking] if they work in a browser: since testers will catch the mistakes, why bother?

First, redefine "done" as, "the change is ready to go out into production, for use by real customers." Second, make each developer responsible for management of the process that gets any particular change to the "done" stage. That includes ensuring that whatever testing the developer considers necessary is completed, whether that be by writing and running automated tests, sending code and test instructions to human testers and reviewing the results, or anything else they can think of.

If a change is found to be defective in some way after the developer has declared it "done," that's the time to sit down with the developer and do some review. This review should be between the developer and one or more senior members of the development team. (It might or might not also include someone from the product owner side; see below.) This review should not be confrontational and certainly should not start out by assuming something was done wrongly, but examine the impact of the released (or about to be released, if you caught it in time) defect and consider what cost-effective measures could have been taken to avoid that happening, i.e., what the developer (and the development team) should change to improve things. (The answer may well be, "Nothing. The cost of preventing all or most things like this is more than the cost living with them.")

The product owner in such meetings, if present, is not there to push the team for technical or procedural changes but for two other things: 1. to provide information about the cost of the negative business impact from the release (or estimated cost, if the defect got caught in time), and 2. to evaluate whether or not the product owner wants to pay additional costs to prevent this kind of problem. (E.g., the developers say, "we can reduce the incidence of this kind of thing by 50% but it will slow development by 5%"; the product owner can then decide whether the increased quality at release is worth the slower rate of getting features rolled out.)

Developers Manage Testing

Sending changes out to human testers is, at a high level, not much different from writing automated tests. There's some sort of script for the testers or test system to follow (even if it's a simple as, "go to this URL and tell me what you think") and the developer evaluates the results to decide whether she's done or she needs to do more work (be that changing code or running more tests).

This script needs to be kept somewhere where it can be reviewed by other developers at (or in preparation for) the meetings I described above. And the history of changes to it needs to be kept as well, so that e.g. if a developer removed something from it (perhaps because she added automated tests to cover something that was problematic or taking too long for the human testers to do) the review can examine that decision. (Personally, I would usually keep this information in the source code repo itself, in text or markdown files, but the developers should do whatever works well for them and the test team.)

The initial script would probably include the story from the product owner, instructions about how to set up anything needed for testing that bit of code, and any other notes the developer finds useful to throw in.

As with anything, developers not experienced with this will probably need some support from others in the development team. You wouldn't expect a developer with little experience in unit testing to be able to write good unit tests right off the bat, or even figure out what needs to be tested; the same goes for managing manual testing.

But the developers need to keep their eye on the goal here: to be able to declare something as "done" with the meaning that they think it will pass all acceptance tests done by the product owner, or that would be done by the product owner if he didn't trust the developers so much, and even any tests that the product owner might think of after the developer's got the story and started coding it.

Acceptance

There's something I somewhat elided above, which is that there really are two "done" stages, under the control of separate teams. The first is the development team's "done" stage, which means "I expect that this will pass all acceptance tests," as mentioned above. The second is the product owner's "I accept this change; let's release it." How much work the product owner does on this side depends on how good a job the development team is doing. Where a development team has proven themselves very reliable at interpreting the requirements and writing code that meets them, the product owner may do minimal testing. Where the development team is not doing such a great job at this, the product owner will have to more verification.

Where you currently stand on this spectrum is not too important: what is important is that, when acceptance testing fails, the development team reviews how it failed (i.e., why they incorrectly felt is was "done" when it wasn't) and develops systems to prevent future similar failures at acceptance testing. Improving this in any particular instance may be a purely internal technical matter for the development team to deal with, or some failures may be due to communications problems between the product owner and the development team, in which case they need to work together to figure out how to fix that.

Summary

Writing automated tests and sending code off to a test team are simply two different methods of acheiving the goal of ensuring a change is of sufficient quality to release. They're almost invariably used together, and the decision about how much of each to use needs to fall to the developer making the change, because it's intimately affected by how she chooses to write her code. So she should be manging that, and needs whatever support is necessary to get good at managing that.

Answer 9

Reviews may work better in that environment.

As a developer I feel the same. Writing tests is dead boring (particularly unit tests). And they are not always effective (particularly unit tests). It depends on the type of logic you write and on the kind of input you get. 100% specified requirements are not that common in my experience so what would your tests prove, really? Requiring people to write unit tests just because can be demotivating. So can shaming people who produce the most "bugs" (they may well be the same ones producing the most value).

Peer pressure is more effective. You have to get used to one another and this may hurt a bit but in the end you will have a team. You will not get a team by imposing external rules that are not supported by the people that have to follow them.

I would even suggest to ditch the off-shore test guys altogether. I mean, why would I try hard to deliver flawless behavior when I am not trusted to do so anyway and there is another layer of verification going over my work anyway? Apparently this is the way we work, right? While they are there, let them make my life a little easier and find the loose ends for me to fix, so I don't have to put too much effort in it.

If that safety net weren't there... I would be more careful. I would feel respected and responsible. With an attitude like "you are going to deliver crap so we will have everything you produce scrutinized", I would not feel inclined to try very hard.

Answer 10

Our organization solved the problem of developers never actually testing their changes themselves by making that part of the process:

1. Giving a change to testing requires that you provide documentation of your changes.
2. Documentation must be made using the standardized template.
3. That template includes the following mandatory sections:
   • What was the reason for the change?
   • What was changed?
   • A "Functionality Test" where the developer does a basic test of the change, documented its execution and results with screenshots in a way which shows that the goal of the change was achieved. This test is not expected to be as thorough as a test performed by the actual QA team. It is just supposed to demonstrate that it now appears to work as far as the developer is concerned.

That way every developer is forced to run their changed code at least once.

Answer 11

You need a "definition of done"

Generally accepted 'definitions of done' include not only working code, but working unit tests, and documentation. Until those things are completed, the task is not done and the code cannot and should not be released to the testing department or god forbid, live.

While you'll often find the concept under the Agile/Scrum umbrella, that working framework isn't necessary to implement this idea.

https://www.scruminc.com/definition-of-done/

https://www.agilealliance.org/glossary/definition-of-done/

The testers are there to catch the things you didn't think of, weird use cases, or things honestly missed.

Answer 12

It seems that the policy went from one extreme to the other. I've never seen working in practice a test coverage of 100%. The trade off between the amount of time and detail required to write a test for a function and the benefit of the test itself is usually very variable. Mandating in a blunt manner a 100% test coverage without considering the trade off will eventually force the developers to take more time than needed to write some tests.

A flexible approach would be better. Require a minimum of test coverage and tell the developers to try and judge how difficult would be to write the test for a function. However another person who is not the developer himself should do some review, check which functions are subject to frequent changes or have such complex requirements that they could be broken easily and flag those functions as the ones that need a test no matter what.

Another thing that would help is finding the developer most skilled with the testing tools and ask him to develop some templates for the most difficult part. Often finding how to build the mocks around a particular function really takes a lot of time. A set of patterns/solution close at hand would save a lot of time.

Answer 13

There's a very simple solution.

You call a meeting with your developers. In that meeting you tell them: "We have a problem with our process right now. It seems that at the moment, the code that is submitted very often does not work properly. That causes QA to spend too much time writing bug reports, and then costs development time fixing those bugs. We had the problem previously that development was extremely high quality, but at huge cost. We need to get more to a point in the middle, where the developers spend a bit more time to make sure there are not too many problems passed to QA, so we can work more efficiently overall".

Some answers are too focussed on the difficulty to find the best point. That's not required. You had a point at one extreme which was inefficient, you have a point at the other extreme now which is inefficient, going further to the middle will be an improvement. Don't ignore an improvement just because you cannot find the optimal point.

Answer 14

You have just answered your questions, when have the two topics been published: "Programmers are reluctant to test code, because (1) they find it just boring, and because (2) if they don't test code, it looks like they deliver the feature faster." What does it mean? How the delivery is done, if there are no tests, which are planned? Any project/module can be delivered (even put to a GIT) only after Maven or Gradle or whatever else has executed all the tests along the build. What is an input of the development of a unit? Analysts or architects must provide a list of cases, which must be subject to tests for a component, or in the worst case, testing engineers (who are responsible for tests) first put tests into the repository and developers get them upon pull request. Maven/Gradle will do the work. Besides, before delivery is done, it HAS(!!) to pass a code review with first attention to the test. And only then, after verifying the consistency of the tests, changes can be merged with the branch, which is used for the industrial build. What is done faster, if there is no proof of work:- bugs and problems? The rule is simple. No tests- no delivery, no work is not done! That's all.

"Spend time pair programming with those who performed the worst, trying to understand why are they so reluctant to test their code, and showing them that it's not that difficult." Really, XP's element as Pair Programming is not dedicated to displaying ABC to developers, which are reluctant to do a part of the work, which they are paid for. Testing - is not only testing, it is a part (an important) of their work, that must be in the definition of a position. The approach you have mentioned- is a spent time on the highest chance of bugs. Your environment, including DevOps, must be built in accordance with the principle: only a component, which passes all the PREDEFINED tests is to go. Otherwise- it is not delivered.

Answer 15

You'll need to go back to developers doing testing.

Where I work, every change pushed out by our team is tested by two other people on the team. The originating developer provides a test plan, and the two peer testers run the test, any other tests they feel like running, and do code review as well.

If it passes this, then our QA department gets a hold of it.

The only drawback I've encountered with this strategy is that some of our developers would be doomed if their lives ever depend on their ability to write clearly.