13

Basically, we have three main projects, two of them are web services, and the other is a web application. While I'm satisfied with covering as much as we can of our web services with functional tests (all three projects have their proper unit tests), functional tests for the web application are taking a lot of developer time to get implemented. By a lot I mean two times, or sometimes more, the time that takes to implement the functionality being tested with unit test included.

The manager policy is to test every single functionality we add, even if is not business critical (i.e a new C.R.U.D).

I do agree with testing all of the web services functionality, because is hard to manually test them, and also, this tests run fast and don't take to much to implement.

So, what's the value in spending more time writing functional test, than writing system code, unit test and fixing QA tikets? Is this normal? Shouldn't we be writing functional tests only for critical functionality and let QA to do regression tests over no critical functionality?

Note: we are not developing medical software or NASA software or nothing that critical.

Improve this question
4

6 Answers 6

Reset to default
17

Functional tests are very important. Yes, they take time to write but if you are writing the right functional tests, they will be more than worth it.

There are a few good reasons to do automated functional tests on an application.

  • When a new feature is added to your web site, it let's you know right away if changes made for that new feature break any other functionality on your site.
  • It's documented knowledge of how the application runs and works together to achieve the business requirements.
  • When it's time to update a 3rd party library, you can update it and run your functional test suite to see if anything breaks. Instead of having to go through every page yourself, you can have a computer do it for you and give you a list of all the tests that broke.
  • Load testing! You can simulate thousands of simultaneous users all hitting your site at once and you can see where your site slows down and buckles under the pressure. You can see how your web site behaves long before you get a late night call that the site has crashed.
  • Functional testing takes time to do manually. Yes, it takes long to write the cases, but if you had to sit down with a binder with 500 pages of tests that you had to complete before you could ship the product you'd wish you had the automated tests!
  • Testing documents get out of date fast. When a new feature is added, you have to make sure to update the master testing document. If someone skips some tests you all of a sudden get bugs creeping into pages that are "done and tested". I currently work in an environment like that, and I can assure you, it's a nightmare.

In the end, yes it takes time to write these cases, but you should take pride in writing them. It's your way of proving, beyond a shadow of a doubt that your code works and it works with all the other features out there. When QA comes to you and says there is a bug, you fix it, and then add it to your test suite to show that it's fixed and make sure it never happens again.

It is your safety net. When someone goes in and hijacks a stored proc and makes a small change so it'll work with their code, you'll catch that it has broken 3 other features in the process. You'll catch it that night and not the night before the deadline!

As for writing functional tests for only system critical functions. That won't give you the whole picture and it'll allow bugs to sneak through. All it take is for one little feature to be added that isn't system critical, but interacts indirectly with a system critical function and you have the potential to have a bug introduced.

Improve this answer
5
  • thanks for your answer. I am aware about the importance and benefits of functional tests, my concerns are about the cost-benefit of testing ALL. We were developing functional test for the last three years, but now in this project, I feel that the cost of implementing test ALL is far much more than finding a bug in production, rising a ticket, and fixing it... I wonder if exist some circumstances where NOT making a functional test is better (in terms of cost-benefit) than not making it, and I wonder if we are in such circumstance, where is better to choose wisely what to test.
    – Pablo Lascano
    Feb 5, 2013 at 21:37
  • @donsenior ~ if you feel it is taking too long to test, then look at the tools you are using. Are you using them properly? Are you even using time saving tools? Writing tests does take longer than writing the code b/c you have more code to write. That's the nature of testing. If you start picking and choosing what to write tests for, then it'll get to the point that no one will write tests, or those tests will be sloppy.
    – Tyanna
    Feb 5, 2013 at 21:51
  • my idea for picking what to test, is not a random pick, but pick the most critical business functionality (and this would not be developers decision, but manager's). And I think just the opposite, developers tend to write sloppy test now because they have to test all, even that functionality that takes five minutes for QA to test and two days for developer to automatize. I do agree that maybe the tools we are using are not the best for testing our web app (fitnesse and java). I'm afraid we are approaching the point of writing and maintaining the functional test more than the system
    – Pablo Lascano
    Feb 5, 2013 at 22:06
  • @donsenior ~ Sure, it takes QA 5min to test a case, but it takes a computer less than a second to test it. You should be asking "Why does it take 2 days to write something that takes 5min to test by hand"? Again, look at your tools. Perhaps QA should be writing some test cases as well? The problem isn't writing test cases for your system, it's how those test cases are being written.
    – Tyanna
    Feb 6, 2013 at 16:50
  • well this tests takes a lot more than a second to run (remember this are functional tests, not unit tests). But that's not a problem, they run at night. I think you are right in that QA should be writing some tests cases as well, but sadly that is not a decision I can make. Thanks a lot for your answers, I've marked this one as accepted!
    – Pablo Lascano
    Feb 6, 2013 at 22:19
8

More than 2 times ... seems a bit much to me. You might want to analyze the reasons for this, they could include:

  • bad tool support for creation and maintenance of the tests

  • contracts of the web services is not sufficiently described in the design. Developers need to work out the contracts while testing, which is usually a time consuming alignment process.

Talk to your developers.

Assuming you are developing in sprints, having these functional tests if just part of the sprint. It ain't done without these tests. If you don't have it, your time for integration testing after the development phase might double.

Improve this answer
2
  • I agree, maybe we are not using the right tool for testing the web app. No problem on testing our web services though. Anyway, besides the right or wrong of how we implement the tests, I am concerned about the costs. I think that at this point, is better (in terms of cost/benefit) to let some test for the QA department and fix the bugs even if they are found in production.
    – Pablo Lascano
    Feb 5, 2013 at 21:49
  • You left out poorly designed classes as the possible reason for taking too long to test. This is by far the most common reason I see when people take forever testing their code.
    – Dunk
    Feb 6, 2013 at 14:44
5

Is spending more time implementing functional test than implementing the system itself normal?

Absolutely. Writing really good tests is likely to take the majority of the time in many (good) shops.
So a 2-1 ratio is fine. Less experienced developers themselves often don't take all the time for tests into account.

Improve this answer
2
  • I am reading this at 2023, is it still valid?
    – Darius.V
    Jul 4 at 18:21
  • 1
    Yes still true in 2023. Still a constant topic in the companies I work in. The main change since 2013 is that now it is tests and also infrastructure such as CI
    – Michael Durrant
    Jul 4 at 22:40
2

There is the law of diminishing returns. Assuming you write tests for the riskiest code first, the value generated by further tests diminishes over time.

Unit tests are code, so they will contain bugs (just like all other code). Fixing those bugs takes time.

In my experience unit-tests contain far more bugs than the system they are testing, and fixing these is a continuous burden.

Improve this answer
1

This is about quality.

If you need to get the market - you will develop your app as quickly as possible. You can even don't have automatic tests at all =) but you will give your app to your auditory before your competitors.

But if you know that your auditory won't go away you will do anything you can not to disappoint them. Every bug ticket will bring down your reputation. Imagine that one bug will remove 50 percent of your reputation, the next - another 25 percent and so one. So can there be too many tests?

Improve this answer
2
  • well I'm not really sure if at this point we are actually reducing tickets. Maybe we have reduce (but not much) QA tickets, but the rate of bugs found by this test are not big enough (over my point of view) to justify such cost of having 2/3 of software engeniers time developing functional tests.
    – Pablo Lascano
    Feb 6, 2013 at 22:30
  • I remember how linked in looked few years ago - lot of visible bugs often. But it still is probably the most popular social platform for employees and HRs. I was angry - how can they do such a bad quality software. But there was no alternative. Somehow linked in occupied everyone, and if you go to another platform if such exists, you simply will not have HRs hunting you. I assume. I am even lazy to try other platforms. So really often software get away with bugs. Or windows - really remember windows 98, win xp with viruses and BSOD :D but they occupied users. Now win 10 is good.
    – Darius.V
    Jul 4 at 18:23
0

If by "is it normal" you ask if it is common, no, it certainly isn't. A lot of dev teams have poor test practices (I belong to one) and even quality books I've read advice to spend roughly as much time coding the functionality than the tests. If by normal you ask if it is healthy, it depends, but two times more tests than needed is better than no test.

It doesn't have to be critical. When you test a functionality, you test something useful for end users, and it is your responsibility to know (and not guess) it is all the time working correctly. If you need two times more for that objective, then it should be done that way - if possible.

It's also possible your policy is sightly too strict about automated tests, but it's hard to tell without knowing the quality they are aiming at, their ressources, and what else they could alocate it to.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.