The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Hot answers tagged

12

It does not matter if you implement E2E and integration tests or not, you need a backup plan either way. Never expect a system to be bug-free just because it was tested. Thus, in your cost estimation, you do not compare the cost for implementing E2E tests against the costs your backup plan estimates in case of a failure, you compare: Costs for doing E2E ...


10

In the cases in which it is absolutely clear that the issue/root cause is the same (for example crashes with the same traceback occurring in the same conditions, only reported by different customers) yes, closing it as a duplicate is IMHO perfectly OK - tracking the 2 (or more) issues separately is just a waste of time/resources. In other cases marking as ...


7

The ticket should be in the column it belongs in, which sounds pretty stupid to say and is exactly why I'm mentioning it. The ticket was being developed, then handed off to QA who found some defects. The ticket now goes back to an engineer to be fixed. Thus, the ticket should go back into development. Each column has a group of people or teams that are ...


7

The biggest problem I see with this is that you're looking at a mini-waterfall rather than a sprint. The description you're giving above forces the team to deliver all or nothing. What if they have 12 items in the sprint and deliver 9? Does that mean you won't test or release what has been done? If so then I dread your release schedule, every time any team ...


7

Perhaps counter intuitively, automated testing can actually reduce development time vs no testing. So it's a win win. The idea is that the tests contribute on a number of levels Force strict requirement gathering and specification This makes a huge impact on the speed of development. No going back asking for more detail, no misunderstandings, no unneeded ...


6

You want to have a clear view of the issues with a software. If a system has 10 issues, you want 10 open bugs. Not 10.000 open bugs of duplicates. You also want the minimum amount of administration. So closing a bug as duplicate when someone finds out it is, is the sensible thing to do. Time is expensive and you want to spend as little time as possible on ...


5

Software verification and validation are all the activities that ensure that the software fulfils all the needs and requirements for its intended purpose. As such V&V is part of the larger set of Quality Assurance activities (the latter also include activities to improve the quality, beyond the area of quality control to which V&V belongs) The ...


5

There are a number of ways to ease/share the burden: Tools You don't say what languages you're using but there are many tools out there that can do basic analysis for you such as StyleCop, ReSharper, FxCop etc. Code coverage For well tested code, coverage percentage should be in high 80s at least. Peer review Even if you're still the gate keeper, you ...


5

You ask: If you assume that testing/debugging/QA cannot catch even the simplest of bugs, and that you must code in extra features to protect against these simple bugs -- isn't that indicative that there is a deeper issue in the development process? Who's suggesting that these processes or people "cannot catch even the simplest of bugs?" That sounds like ...


4

Well why do you separate dev and test environments? Normally it's because testers want control over when their environment changes. It's annoying when a test that was passing starts failing without you noticing because you didn't realize the environment changed. Now a dev team pushing to a dev environment is not weird at all. It's not weird for the dev ...


4

In Scrum, every sprint ends in a potentially shippable product. If regression testing is needed for a particular build to be releasable from a technical standpoint, then the answer to your question is yes. You will actually have to test much more than what you have said. There might be another solution, but the only way I've ever known to handle this is ...


4

There are two techniques that work together to avoid this type of scenario from occurring: Step 1: write a failing test before fixing the bug When the bug has been found, the first thing to do is to write a test that will pass when the bug is fixed and thus it fails whilst the bug still exists. The dev then fixes the bug and in the process makes the test ...


3

The relationship... doesn’t exist. Modern software development generally doesn’t have discrete phases where you take and validate some input, do some work, and then pass it down the assembly line. And these days even having a separate role for testers or other quality assurance work is more and more rare. At the scale necessary for most software, the ...


3

It's hard to answer "what is typical" because there are different types of customers who have their own demands on how software is deployed in their systems. There is a gamut of what happens from companies that maintain and deploy their own software (like Netflix, AirBnB, Stack Exchange, etc.) to companies that require hand-off to third parties to deploy. ...


2

There is no right answer for this question. This depends on how your team develop new features, how they are delivered on the production, the development cycle of the team, if the project have budget to have many environments, etc. I already work on a software where only some developed features are delivered to tests. To not merge a lot of branches that ...


2

Putting the dev/tester ratio issue aside, since you probably aren't in a position to fix that, the problem seems to be with your feature branches lagging behind master. As Ewan has pointed out, you want to be testing code that is representative of what will be deployed. So, testing on a feature branch that is behind master is not great. What you could do ...


2

I would do QA on the dev (or master in your case) branch rather than the feature branch. Devs should test on the feature branch to ensure the features are complete, but QA should test the product as a whole after merging in the new feature. Otherwise you are testing twice, on the feature branch and after merging Just to do the sums, assuming no bugs found ...


2

I wholeheartedly second David Arno's suggestions about failing tests and code reviews. Also, there are often ways to write the code to make it less susceptible to that kind of error in the future. For example, if the error was created by calling a function before its prerequisites were met, you can split your classes differently so that would create a ...


1

This metric, to determine what percentage of tests pass, doesn't make much sense to me. There are two reasons why a test would fail: The test found an error. Some change in code caused the behavior under test to change and the test is alerting you to a case that needs to be considered. The change needs to be reworked to ensure that the system has the ...


1

It's very tempting to answer your question with "it depends", since that's the answer to many "should I...?" questions. In this case, however, the answer is no. You should not create a sprint test plan. At least, not a formal one. The only test plan you need for a sprint is "verify that every completed story is well tested". As a QA engineer on a scrum ...


1

My answer? Maybe, probably not. EOE tests are good when they are very simple. If you are planning to cover basic scenarios, you can manage to gain some advantage with EOE tests. But if you have a really complex and big application (mission critical or not), this EOE tests will be expensive to maintain and you need to know your scenario to valuate if worth ...


1

I see tests at most complementing the human code review Hmm out of everything you say this stands out to me the most. Maybe there is a nuance to your meaning here, but always remember that the idea is to produce working software, not well coded software. I would suggest the first thing you do is write down the rules that you are reviewing the code against. ...


1

A few things stand out to me. First, the ratio of developers to testers stands out as being very lopsided. In my experiences, a good ratio is about 3-5 developers to 1 manual tester. This seems to work out well as long as the tester is involved from early in requirements development to begin to start identifying test strategies, approaches, and cases. The ...


1

Stop asking QA to test everything all the time. Get the developers to do it themselves. Where I work, QA don't do testing. They are there to ensure that developers are following the correct process. The only time they get involved in testing is the final formal test before software is released to a customer. Even then, they are there to ensure that the ...


1

I'm not sure I understand your specific problem exactly, but the general theme itself is very common - with a number of aspects. Requirements Without clear, documented and agreed targets, the function of any system is up for grabs. In the absence of requirements there are likely to be a lot of differences of opinion as to how a system should work with the ...


1

It would have been common sense that you ask the developer if there was a particular reason to test in the dev environment. If there is a good reason you test in the dev environment. If the devs say there is a good reason but you don’t see the reason, you have the choice between testing in the dev environment and possibly starting a fight. If the devs admit ...


1

General Advice The closer the environment is to a clone of production, the more accurate your testing will be. The further away it is, the more caveats. Many development environments are not similar to production. Mostly because they are playgrounds for developers to trial different configurations, software bundles, services, etc... This doesn't mean they ...


Only top voted, non community-wiki answers of a minimum length are eligible